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Introduction 

This  evaluation  study  represents  one  part  of  a  project  sponsored  by  the  Army 
Research  Institute.  The  gok  of  the  project  is  to  derive  a  model  to  guide  the  evaluation  of 
team  performance.  The  purpose  of  this  article  is  to  describe  a  model-based  performance 
measurement  instrument  that  has  been  designed  to  assess  and  diagnose  team  skills  and  to 
describe  the  findings  of  an  initial  evaluation  of  that  instrument.  Version  2.0  of  the  Advanced 
Team  Decision  Making  (ATDM  2.0)  model  (see  Thordsen,  Klein,  &  Kyne,  1994)  served  as 
the  basis  for  the  instrument. 

ATDM  2.0  is  a  cognitive  model  of  team  performance  in  that  the  metaphor  underlying 
the  model  is  that  of  individual  cognition.  The  model  views  a  team  as  an  intelligent  entity, 
capable  of  learning  and  adapting,  making  decisions,  reacting  to  surprises,  and  planning 
and  executing.  The  ATDM  2.0  model  describes  four  components  of  team  performance: 
team  resources,  team  identity,  team  cognition,  and  team  self-management.  Each  of  these 
components  is  divided  further  into  behavioral  dimensions. 

The  model  is  a  general  one  in  that  it  is  not  domain  bound,  but  applies  to  teams 
functioning  in  virtually  any  domain.  This  flexibility  is  achieved  by  organizing  the  model  into 
four  tiers.  The  two  highest  tiers  (illustrated  in  Figure  1)  represent  the  general  components 
’  and  behavioral  dimensions  of  team  decision-making  and  performance.  These  two  levels 
describe  aspects  of  proficient  team  decision-making  and  performance,  regardless  of  the 
domain.  In  contrast,  the  two  lowest  tiers  of  the  model,  the  specific  behaviors  and  anchors, 
are  specific  to  any  one  domain.  These  are  the  specific  observable  behaviors  that  are 
indicative  of  either  good  or  poor  performance  for  each  of  the  behavioral  dimensions.  Only 
the  upper  two  tiers  (Team  Components  and  Behavioral  Dimensions)  of  the  model  are 
diagrammed  in  Figure  1.  The  lower  two  tiers  cannob  be  represented  in  the  diagram  since 
they  vary  depending  upon  the  domain  of  interest. 

One  purpose  of  the  ATDM  2.0  model  is  to  inform  us  about  team  processes.  But 
more  importantly,  it  is  intended  to  serve  as  the  basis  for  diagnosing  team  processes  and  for 
developing  interventions  that  will  enhance  team  performance.  To  meet  these  goals,  it  is 
essential  to  have  the  capability  to  assess  and  diagnose  team  processes  as  the  team  trains  and 
functions  within  its  operational  setting.  Thus,  the  objective  of  this  portion  of  the  project  was 
to  develop  a  draft  team  performance  measurement  instrument  based  on  the  ATDM  2.0  model 
and  to  evaluate  the  utility  of  the  instrument  in  a  field  setting.  This  paper  describes  the 
results 
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Figure  1.  Advanced  Team  Decision  Making  Model,  Version  2.0. 

of  this  work:  how  we  developed  the  measurement  instrument,  how  we  evaluated  the 
’  instrument  in  a  field  study,  and  finally  our  conclusions  and  revisions  to  the  instrument  based 
on  the  findings  of  the  field  study. 

Instrument  Development 

The  two  upper  tiers  (Team  Components  and  their  Behavioral  Dimensions)  of  the 
model  are  descriptive  of  teams  in  general,  i.e.,  they  are  not  domain  specific.  However,  the 
lower  two  tiers  (Specific  Behaviors  and  their  Anchors)  serve  as  a  bridge  between  this  general 
description  of  teams  and  specific  teams  in  specific  domains.  If  we  can  identify  the  Specific 
Behaviors  and  Anchors  for  a  particular  team,  we  have  the  basis  for  an  assessment 
instrument.  This  instrument  can  then  be  use  to  measure  and  evaluate  aspects  of  the  team’s 
decision-making  and  performance.  In  this  section  we  will  describe  how  we  used  the  upper 
generic  tiers  of  the  model  to  guide  us  in  identifying  the  specific  behaviors  and  anchors  for  a 
particular  domain  and  how  we  then  used  these  specific  behaviors  and  anchors  as  the  basis  for 
an  assessment  instrument. 

We  established  several  criteria  or  goals  for  a  successful  assessment  instrument. 
Obviously,  we  wanted  to  develop  an  instrument  that  could  be  implemented  reliably  by 
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researchers  and  experienced  domain  personnel.  In  addition,  we  wanted  to  ensure  that  the 
instrument  was  sensitive  to  changes  in  team  performance.  We  also  established  several 
functional  requirements  for  the  instrument.  First,  the  instrument  had  to  be  flexible  so  that  it 
could  be  adapted  from  one  domain  to  another  relatively  easily.  Second,  we  wanted  an 
instrument  that  could  be  handed  off  for  use  by  domain  personnel.  We  wanted  the  ins^ment 
to  be  used  by  people  sophisticated  in  the  domain,  and  not  just  by  the  people  involved  in  the 
model  development.  Finally,  the  instrument  had  to  be  useful  for  identifying  the  strengAs 
and  weaknesses  of  teams  as  they  performed  their  jobs  in  operational  settings.  We  ^nsidered 
these  goals  throughout  the  development  process  and  used  them  as  criteria  when  designing  the 
evaluation  of  the  instrument. 

As  described  by  Thordsen,  Klein,  and  Kyne  (1994),  the  two  higher  tiers  of  the 
ATDM  2.0  model  are  general.  The  components  and  behavioral  dimensions  describe  key 
elements  of  team  performance  in  many  domains.  To  apply  these  concepts  usefully  in  a  given 
domain,  we  must  identify  specific  behavioral  examples  of  how  team  performance  is 
instantiated  in  the  domain.  The  model  must  be  anchored  in  the  domain.  What  constitutes 
good  and  poor  i)erformance  on  each  of  these  dimensions  and  components  within  the  domain? 
These  examples  or  anchors  are  at  the  core  of  the  performance  measurement  instrument. 
Therefore,  identifying  specific  behavioral  markers  was  the  key  to  the  development  of  the 
instrument. 

Our  initial  efforts  at  developing  an  instrument  focused  on  the  Army’s  battalion  battle 
staff.  This  is  a  large,  diverse  team  that  performs  both  planning  and  execution  functions 
under  conditions  of  ambiguity,  time  pressure,  and  high  risk.  In  addition,  the  t^  has  an 
overt  hierarchical  command  structure.  Our  goal  was  to  identify  specific  behavioral  markers 
related  to  team  proficiency  that  indicated  good  and  poor  performance  on  each  of  the 
behavioral  dimensions  and  components.  Our  criteria  were  for  these  markers  to  be  observable 
and  to  serve  as  indicators  of  the  quality  of  team  performance  for  experienced  observers. 

To  identify  these  markers,  we  enlisted  the  support  of  two  domain  subject  matter 
experts  (SMEs).  These  SMEs  had  served  as  battalion  commanders  and  had  significant  active 
duty  experience  with  battalion  and  brigade  battle  staffs.  Researchers  conducted  numerous 
extensive  interviews  with  each  SME  to  elucidate  the  ATDM  2.0  model  and  to  elicit 
information  about  the  battle  staffs  tasks  and  indicators  of  performance  quality.  As  the 
model  took  shape,  the  SMEs  were  asked  for  comments  and  suggested  revisions.  As  such, 
the  SMEs  helped  shape  the  ATDM  2.0  model  as  well  as  identify  behavioral  markers.  This 
process  resulted  in  a  detailed  listing  of  behavioral  markers  that  represented  effective  and 
ineffective  battle  staff  performance  for  each  of  the  behavioral  dimensions  incorporated  into 
the  model.  These  behavioral  markers  bridged  the  model  to  the  battle  staff  domain.  The 
specific  behaviors  for  each  behavioral  dimension  are  included  in  attachment  A  of  this 
document. 

We  had  planned  to  conduct  the  field  study  with  Army  Battalion  battle  staff  during 
training  exercises  at  their  home  stations  and  then  again  at  the  National  Training  Center,  Fort 


3 


Irwin,  CA.  This  would  have  given  us  the  opportunity  to  test  the  instrument’s  capability  for 
change  in  team  performance  as  the  battle  staff  gained  experience.  Because  we  were 
unable  to  secure  access  to  these  test  sites,  we  switched  to  the  Incident  Command  Structure  of 
the  firefighting  domain. 

The  early  development  of  the  model  and  the  measurement  instrument  with  the 
battalion  battle  staff  served  as  a  template  for  the  measurement  instrument  to  be  evaluated  in 
the  field  study  »ging  urban  firefighting  teams.  We  believe  that  there  are  certain  similarities 
between  battle  staffs  and  firefighting  teams.  For  example,  both  teams  execute  plans  under 
time  pressure,  ambiguity,  and  high  risk.  Furthermore,  both  types  of  teams  employ 
hierarchical  command  structures.  Therefore,  we  simply  needed  to  revise  the  behavioral 
markers  to  establish  a  set  that  is  valid  for  firefighting  teams  and  to  create  a  useable  format 
for  the  instrument. 

To  accomplish  these  goals,  we  conducted  a  pilot  study  in  an  urban  fireground  setting. 
Two  researchers  each  spent  six  days  with  the  fire  department  of  a  large  U.S.  city.  They 
accompanied  nine  Battalion  Chiefs  (BCs)  on  a  total  of  20  incident  calls.  Using  the  revised 
instrument,  researchers  observed  the  firefighting  teams  in  action  and  interviewed  the  BCs  and 
other  members  of  the  teams  both  during  and  after  the  incidents.  The  pilot  study  gave 
researchers  the  opportunity  to  determine  the  similarity  and  relevance  of  the  instrument’s 
markers  and  whether  it  could  be  used  easily  in  this  setting. 

Several  interesting  findings  emerged  from  this  pilot  study.  First,  the  13  behavioral 
dimensions  that  make  up  the  4  team  components  of  the  current  ATDM  2.0  model  appear  to 
be  domain  independent.  That  is,  they  represent  critical  elements  of  team  performance  across 
domains.  The  researchers  did  not  recognize  any  team  components  or  behavioral  dimensions 
as  inappropriate  for  this  domain.  In  fact,  when  BCs  described  what  they  considered  to  be 
’important  factors  for  their  teams,  these  factors  matched  many  of  the  behavioral  dimensions  of 
the  ATDM  2.0.  Every  dimension  was  mentioned  in  one  way  or  another  by  the  BCs, 

A  second  finding  of  the  pilot  study  was  that  performance  on  the  various  dimensions 
was  easily  observed  by  the  researchers.  The  BC’s  vantage  point  offered  a  clear  view  of  the 
team’s  performance.  The  researchers’  challenge  was  to  remain  alert  for  the  more  subtle 
behaviors  and  not  to  get  blindsided  by  the  obvious  ones.  Because  of  the  BCs  level  of  domain 
expertise,  they  were  able  to  point  out  the  subtle  behaviors  as  researchers  implemented  the 
measurement  instrument. 

Finally,  many  of  the  BCs  volunteered  to  help  revise  and  edit  the  descriptions  of  the 
behavioral  dimensions  and  the  supporting  list  of  behavioral  markers.  We  submitted  several 
iterative  versions  of  these  items  to  the  BCs  for  their  feedback.  This  resulted  in  a  list  of 
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behavioral  markers  for  each  dimension  that  were  meaningful  for  the  intended  users  of  the 
instrument. 

The  Instrument 


The  instrument  is  designed  to  allow  an  indq)endent  observer  or  a  knowledgeable  team 
member  assess  and  document  how  a  team  performed  during  a  specific  incident.  This  version 
of  the  instrument  is  intended  for  use  by  Battalion  Chiefs  (BC)  in  urban  fireground  settings. 
The  fireground  team  includes  those  under  the  command  of  the  BC  as  the  team  responds  to 
emergency  calls.  The  composition  and  size  of  the  team  depends  on  the  specific  incident. 

The  team  may  respond  to  a  wide  variety  of  situations  including  various  types  and  sizes  of 
fires,  automobile  accidents,  police  assistance,  and  gas  and  chemical  leaks. 

The  instrument  requires  the  observer  to  provide  subjective  ratings  of  team 
performance  at  the  level  of  behavioral  dimensions  specified  by  the  ATDM  2.0  model.  The 
observer  completes  the  ratings  either  as  the  incident  evolves,  if  time  permits,  or  after  the 
incident  when  the  observer  has  time  to  reflect  on  what  occurred. 

The  instrument  comprises  two  sections.  The  first  is  a  single  page  on  which  the 
observer  records  his/her  ratings  for  a  single  incident  (see  Figure  2).  Note  that  this  particular 
version  of  the  instrument  reflects  the  behavioral  dimensions  employed  during  the  pilot  study 
and  as  such  contains  17  rather  than  the  13  of  the  current  version.  In  the  latest  version. 

Range  of  Factors  and  Time  Horizon  are  treated  as  sub-dimensions  of  Team  Focus;  Diverge 
and  Converge  are  sub-dimensions,  along  with  Size  Up,  under  the  dimension  of  Situation 
Assessment;  [envisioning]  Course  of  Action  is  now  consolidated  within  Envisioning  Goals 
and  Plans:  and  Expectancies  is  addressed  as  special  cases  of  either  Situation  Assessment  or 
Managing  Uncertainty,  depending  on  the  context.  This  first  page  contains  space  to  record 
’  information  about  the  original  17  behavioral  dimensions.  The  observer  provides  two  types  of 
information  for  each  dimension.  First,  the  observer  indicates  whether  behavior  indicative  of 
the  specific  dimension  was  observed,  absent,  not  observed,  or  not  applicable  during  the 
incident  by  checking  the  appropriate  box.  The  observer  checks  the  "Present"  box  if  s/he 
observes  behavior  indicative  of  a  specific  dimension.  The  category  "Absent"  is  reserved  for 
those  cases  where  the  observer  is  sure  that  the  behavior  would  have  been  applicable,  but  did 
not  occur.  "Not;  Observed"  indicates  that  the  behavior  may  have  occurred,  but  the  observer 
did  not  see  it  or  is  not  sure. 

The  second  type  of  information  provided  by  the  observer  is  a  subjective  rating  of  the 
quality  of  team  performance  for  those  dimensions  and  components  that  are  positively 
observed  during  the  incident.  For  this  purpose,  the  observers  used  the  rating  scales  in  the 
right  column  of  the  page  (see  Figure  2).  These  scales  are  5-point  rating  scales  with  a  scale 
value  of  one  anchored  to  "Very  Ineffective,"  and  a  scale  value  of  5  anchored  to  "Highly 
Effective."  The  observers  were  instructed  to  use  their  experience  and  knowledge  of  the 
ATDM  2.0  model  to  arrive  at  these  ratings. 
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Date: _ Time: _ Coder: _ Incident  #: 

Incident  Type: _ 


Behavioral  Dimensions 

P 

A 

n/o 

n/a 

L- 

—  >_ 

—  > — 

—  >H 

Member  Resources 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Leader  Resources 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Other  Resources 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Procedures 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Roles  &  Functions 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Engagement 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Compensation 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Envision  Goals 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Range  of  Factors 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Time  Horizon 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Manage  Uncertainty 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Diverge 

□ 

□ 

□ 

n 

1 

2 

3 

4 

5 

Converge 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Course  of  Action 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

’  Expectancies 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Monitor  &  Adjust 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Time  Management 

□ 

□ 

□ 

□ 

1 

2 

3 

4 

5 

Figure  2.  Rating  scales. 

The  instrument  also  includes  a  second  section  which  provides  support  for  the  observer 
as  s/he  observes  team  performance  and  completes  the  rating  scales.  This  section  serves  as  a 
quick  reference  guide  for  the  behavioral  dimensions  of  the  ATDM  2.0  model.  Grouped  by 
component,  each  behavioral  dimension  is  allotted  one  page  which  is  tabbed  for  easy  access. 
This  page  provides  a  definition  of  the  dimension  with  descriptions  of  indicators  of  effective 
performance  and  space  for  the  observer  to  take  notes.  The  observer  can  refer  to  these 
descriptions  before,  during,  or  after  an  incident.  This  facilitates  reliability  among  observers 
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and  for  one  observer  across  different  incidents.  The  information  presented  to  the  observer  in 
this  section  of  the  instrument  is  in  Attachment  B  of  this  report. 

The  Field  Study 

With  the  draft  measurement  instrument  in  hand,  our  next  step  was  to  evaluate  it  in  an 
operational  setting.  The  objectives  of  this  field  study  were  to  determine  the  reliability, 
useability,  and  acceptance  of  the  draft  instrument  within  the  constraints  of  an  operational 
setting.  We  conducted  this  study  with  the  cooperation  of  the  fire  department  of  a  large  U.S. 
city,  the  same  fire  department  that  supported  the  pilot  study  and  instrument  development. 

Note  that  for  the  field  study,  the  instrument  contained  16  behavioral  dimensions  with  the 
earlier  dimensions  of  Diverge  and  Converge  being  handled  as  sub-dimensions,  along  with 
Size-up,  of  Situation  Assessment. 

Two  researchers  accompanied  a  total  of  10  BCs,  six  of  whom  had  participated  in  the 
pilot  study,  two  had  not.  We  rode  with  the  busiest  Battalions  and  accompanied  BCs  who 
were  on  the  busiest  shifts.  We  observed  a  total  of  36  incidents.  These  incidents  were 
response  calls  to  a  variety  of  emergencies  including  fires,  assist  police,  and  automobile 
accidents.  The  teams  ranged  in  size  and  composition  depending  on  the  nature  of  the 
emergency.  Some  teams  included  a  single  response  unit  such  as  a  rescue  unit  and  the 
battalion  chief.  Others  were  multiple  alarm  fires  that  included  more  than  one  company  and, 
in  some  cases,  more  than  one  battalion. 

The  researchers,  situated  in  the  fire  house,  responded  to  calls  with  the  BC.  They 
remained  with  the  BC  throughout  each  incident.  The  researchers  queried  the  BC  and  his  aide 
during  the  course  of  each  incident,  monitored  radio  communications,  observed  team 
members’  actions,  and  interviewed  team  members  (including  the  BC)  following  each 
incident.  In  addition,  each  researcher  completed  a  rating  form  and  took  comprehensive  notes 
for  each  incident.  To  the  degree  possible,  they  completed  the  rating  scales  as  the  incident 
evolved;  in  some  cases,  they  completed  the  scales  following  the  incident.  The  researchers 
arrived  at  their  ratings  independently;  they  did  not  collaborate  while  completing  the  rating 
scales. 


The  ratings  provided  by  the  researchers  posed  an  opportunity  to  assess  the  reliability 
among  raters  who  are  intimately  familiar  with  the  ATDM  2.0  model  and  team  performance 
in  other  domains.  However,  it  did  not  provide  an  opportunity  to  assess  whether  the  BCs  saw 
the  incidents  in  the  same  way  as  the  researchers  or  whether  they  would  apply  the  instrument 
in  the  same  way  as  the  researchers.  To  answer  these  questions,  we  asked  the  BCs  to 
complete  a  rating  form  for  each  of  the  incidents  in  which  they  participated.  These  ratings 
were  provided  after  the  incidents  had  occurred.  The  reason  for  this  was  to  use  these 
incidents  to  instruct  the  BCs  on  the  behavioral  dimensions.  Talking  to  the  BCs  about  the 
instrument  during  real-world  incidents  was  the  most  efficient  way  of  getting  the  BCs 
prepared  to  use  it.  The  BCs  were  provided  written  descriptions  of  each  of  the  incidents  as 
well  as  a  measurement  instrument.  They  completed  the  rating  forms  independently.  With 
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these  data,  we  assessed  the  inter-rater  reliability  among  the  BCs  and  the  two  researchers  for 
each  of  several  selected  incidents.  These  data  are  described  below. 

P(»1iahi1itv  between  researchers.  For  each  incident,  the  measurement  instrument  asked 
the  observer  to  make  an  initial  judgment  about  each  behavioral  dimension.  The  observers 
indicated  one  of  four  categories: 

•  Observed:  whether  they  observed  behaviors  indicative  of  the  dimension 

•  Absent:  whether  the  behavior  should  have  occurred,  but  did  not 

•  Not  Observed:  whether  behavior  might  have  occurred,  but  they  did  not  observe  it 

•  Not  Applicable:  whether  the  dimension  was  not  applicable  for  the  particular 
incident. 

Neither  of  the  two  researchers  used  the  categories  "Absent"  or  "Not  Applicable"  to 
describe  the  16  behavioral  dimensions  for  any  of  the  36  incidents.  Both  researchers  indicated 
that  the  dimensions  were  either  "Present"  or  "Not  Observed."  To  assess  inter-rater 
reliability  for  making  these  categorizations,  we  calculated  the  percent  of  agreement  among 
the  two  observers.  For  each  behavioral  dimension,  the  frequency  of  agreement  among  the 
two  observers  was  divided  by  the  total  number  of  opportunities  to  agree  (36).  Table  1 
presents  these  data. 

Generally,  there  was  good  agreement  among  the  two  researchers  about  their 
categorizations.  The  percent  agreement  across  all  dimensions  and  incidents  was  83.1%. 
However,  there  was  variability  across  the  various  dimensions.  Clearly,  the  highest 
’  agreement  among  the  two  observers  concerned  the  four  dimensions  of  Team  Resources  and 
the  dimension  Roles  and  Functions  under  Team  Identity.  There  were  no  disagreements 
among  the  observers  for  these  five  dimensions.  The  least  agreement  occurred  for  the 
dimension  Course  of  Action  (52.8%  agreement)  followed  by  Range  of  Factors  (58.3% 
agreement). 

Another  interesting  point  concerns  the  pattern  of  disagreements  among  the  two 
observers.  Since  neither  observer  used  the  categories  "Absent"  or  "Not  Applicable," 
disagreements  occurred  only  when  one  researcher  indicated  "Observed"  when  the  other 
indicated  "Not  Observed."  In  77%  of  these  disagreements.  Researcher  1  indicated 
"Observed"  and  Researcher  2  indicated  the  reverse.  This  suggests  that  the  two  researchers 
were  systematically  applying  different  sets  of  rules  for  making  the  categorizations.  See 
Table  2  for  the  number  of  incidents  each  researcher  observed  a  specific  behavioral 
dimension. 
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Table  1 


Percait  Agreement  Among  Two  Observers 


Dimension 

Number 

Disagreements 

% 

Agree 

Member  Resources 

0 

100 

Leader  Resources 

0 

100 

Other  (Non-Human)  Resources 

0 

100 

Procedures 

0 

100 

100%  Total  percent  agreement 
for  Team  Resources 

Roles  and  Functions 

0 

100 

Engagement 

6 

83.3 

Compensation 

10 

72.2 

85.2%  Total  percent  agreement 
for  Team 

Identity 

Envision  Goals 

13 

63.9 

Range  of  Factors 

15 

58.3 

Time  Horizon 

6 

83.3 

Manage  Uncertainty 

7 

80.5 

Situation  Assessment 

1 

97.2 

Course  of  Action 

17 

52.8 

Expectancies 

7 

80.5 

73.8%  Total  percent  agreement 
for  Team  Cognition 

Monitor  and  Adjust 

13 

63.9 

Time  Management 

2 

94.4 

79.2%  Total  percent  agreement 

for  Self  management 
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Table  2 

Number  of  Times  Researchers  Observed  Behavioral  Dimension 


Behavioral  Dimension 


MR 

LE 

NHR 

P 

R&F 

E 

C 

EG 

RofF 

TH 

MU 

SA 

CoA 

E 

M&A 

TM 

R1 

36 

36 

36 

36 

36 

36 

7 

1 

8 

6 

10 

35 

8 

10 

15 

1 

R2 

36 

36 

36 

36 

36 

30 

17 

13 

19 

8 

10 

36 

25 

15 

25 

1 

The  researchers  discussed  the  reasons  for  their  differences  in  ratings.  They  found 
that  one  researcher  would  indicate  on  the  rating  sheet  that  he  observed  a  behavioral 
dimension  if  he  knew  it  had  been  taken  into  account  prior  to  execution  of  the  task.  If  the 
other  researcher  did  not  observe  the  behavioral  dimension  during  task  execution  then  she  did 
not  indicate  it  on  the  rating  form  even  though  she  agreed  it  had  been  taken  into  account  prior 
to  task  execution.  An  example  of  this  is  the  behavioral  dimension  range  of  factors, 
representing  the  cognitive  component  of  the  ATDM  2.0  model.  Many  factors  are  taken  into 
account  prior  to  an  emergency  and  a  plan  of  action  is  developed  to  deal  with  them. 

’We  were  interested  in  the  amount  of  variability  in  agreement  across  the  four  team 
components.  This  comparison  was  made  by  averaging  the  percent  agreement  between  the 
two  observers  across  all  of  the  dimensions  that  constitute  each  team  component.  These  mean 
percent  agreements  were  100%  for  Team  Resources,  85.2%  for  Team  Identity,  73.8%  for 
*  Team  Cognition,  and  79.2%  for  Self  Management.  Clearly,  behaviors  relevant  for  Team 
Resources  were  relatively  easy  for  the  researchers  to  observe.  The  cognitive  aspects  of 
performance  were  less  obvious. 

For  each  dimension  that  the  researchers  rated  "observed,"  they  also  provided  a  rating 
of  the  team’s  quality  of  performance  on  the  dimension.  Ratings  were  provided  on  a  5-point 
scale  with  a  scale  value  of  one  anchored  to  "Very  Ineffective"  and  a  sc^e  value  of  five 
anchored  to  "Highly  Effective."  Comparisons  of  these  scores  also  provWe  some  indication 
of  inter-rater  reliability.  Because  both  researchers  did  not  observe  all  dimensions  during 
each  incident,  the  number  of  pairs  of  scores  compared  are  not  equal  for  all  behavioral 
dimensions.  Comparisons  were  made  only  for  those  incidents  in  which  both  observers  rated 
the  dimensions  "observed."  Two  dimensions,  Time  Management  and  Envisioning  Goals,  did 
not  have  a  sufficient  number  of  pairs  of  ratings  to  make  these  comparisons. 

The  most  interesting  finding  of  this  analysis  was  a  severe  ceiling  effect.  There  was 
very  littip-  variability  in  the  ratings  provided  by  the  two  observers.  In  fact,  both  observers 
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considered  the  teams  to  be  highly  effective  on  every  dimension.  Table  3  presents  the  mean 
and  standard  deviation  of  the  ratings  provided  by  each  observer  for  each  behavioral 
dimension.  All  dimensions  received  mean  ratings  well  above  4.0  with  very  little  variability 
from  either  observer.  Thus,  there  was  a  high  degree  of  reliability  among  the  two  ob^ryers 
in  rating  the  effectiveness  of  tp.am  performance.  In  the  next  section  you  will  find  a  similar 
effect  among  the  BCs  mean  ratings,  but  there  is  a  greater  standard  deviation,  indicating  more 
variability  in  recognizing  the  quality  of  performance  by  people  familiar  with  the  domain. 

This  will  be  explained  further  in  the  findings  and  conclusions  section  of  this  paper. 

Prior  to  acVing  BCs  to  rate  the  performance  of  their  teams  for  the  incidents  observed 
by  the  two  researchers,  the  instrument  was  revised.  Specifically,  two  behavioral  dimensions 
were  removed:  Articulating  Expectancies  and  [envisioning]  Courses  of  Action.  These 
behavioral  dimensions  were  addressed  by  other  behavioral  dimensions  in  the  cognitive 
component  of  ATDM  2.0.  Articulating  Expectancies  is  now  treated  as  special  cases  of  either 
Situation  Assessment  or  Managing  Uncertainty,  depending  on  the  context.  Courses  of 
Action,  is  now  part  of  Envisioning  Goals  and  Plans.  In  the  firefighting  domain,  the  team 
takes  action  directly  from  the  situation  assessment  as  the  incident  commander  watches 
whether  or  not  the  actions  are  achieving  the  objectives.  With  these  changes  the  instrument 
still  addressed  the  critical  dimensions  of  team  decision-making  and  performance.  There  was 
also  a  secondary  benefit  of  reducing  the  number  of  behavioral  dimensions:  it  produced  a 
more  compact  instrument  that  was  more  manageable  for  the  users. 

Pp.liahilitv  among  researchers  and  Battalion  Chiefs.  The  data  described  above  answer 
questions  concerning  how  researchers  who  are  very  knowledgeable  about  the  ATDM  2.0 
model  and  not  so  knowledgeable  about  the  firefighting  domain  would  implement  the 
measurement  instrument.  However,  they  do  not  address  questions  concerning  whether  we 
could  hand  over  the  materials  to  an  expert  in  firefighting  and  whether  these  experts  would 
’  achieve  results  similar  to  those  produced  by  the  two  researchers. 

Therefore,  we  requested  that  six  of  the  battalion  chiefs  complete  a  measurement 
instrument  for  selected  incidents  in  which  they  participated.  The  criteria  for  selecting  these 
six  BCs  were  that  we  observed  multiple  incidents  with  each  of  them  and  at  least  one  of  the 
incidents  was  more  substantial  than  a  simple  respond  and  recall.  All  told,  we  obtained 
expert  ratings  for  a  total  of  18  incidents;  however,  for  this  portion  of  the  analysis,  we  did  not 
include  the  data  from  two  incidents.  One  incident  was  not  used  because  researchersdid  not 
observe  the  entire  incident.  The  other  was  not  used  because  the  BC  was  confused  about  the 
rating  scales.  These  data  (incidents  =16)  were  then  compared  to  those  produced  by  the  two 
researchers. 

However,  the  instrument  used  by  BCs  differs  from  the  one  used  by  researchers  in  two 
ways.  The  first  concerns  the  removal  of  two  behavioral  dimensions  from  the  Cognitive 
Component  of  ATDM  2.0  as  described  previously.  The  second  concerns  the  categories  for 
indicating  whether  or  not  a  dimension  was  present. 
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Tables 


Mean  Ratings  of  Effectiveness 


Number  of  incidents 
observed  by  both 


Observer  1 

Observer  2 

researchers 

Dimension 

X 

X 

SD 

N 

Member  Resources 

4.9 

.3 

4.9 

.3 

36 

Leader  Resources 

5.0 

.2 

4.9 

•3 

36 

Non-human  Resources 

4.9 

.6 

4.9 

.3 

36 

Procedures 

4.9 

.3 

4.9 

.3 

36 

Roles  and  Functions 

4.8 

.6 

4.9 

.5 

36 

Engagement 

4.8 

.5 

4.8 

.6 

30 

Compensation 

4.6 

1.1 

4.2 

1.3 

7 

Envision  Goals 

NA 

NA 

1 

Range  of  Factors 

5.0 

.0 

4.8 

.4 

6 

Time  Horizon 

4.5 

1.0 

4.4 

1.2 

4 

Manage  Uncertainty 

4.8 

.7 

4.4 

1.0 

7 

Situation  Assessment 

4.9 

.4 

4.8 

.5 

35 

Coiurse  of  Action 

5.0 

.0 

4.8 

.4 

8 

Expectancies 

4.8 

.7 

4.8 

.7 

8 

Moiutor  and  Adjust 

4.7 

.8 

4.5 

.7 

13 

Time  Management 

NA 

NA 

0 
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We  asked  the  BCs  to  tell  us  if  the  dimension  was  observed  during  the  incident  or 
considered  prior  to  the  incident.  Our  goal  was  to  eliminate  the  problem  identified  in  the  field 
study  where  a  discrepancy  was  found  between  the  two  researchers  concerning  when  a 
behavioral  dimension  was  taken  into  account  (before  or  during  task  execution).  We  asked 
the  BCs  to  indicate  whether  the  dimension  was  present  during  task  execution,  present  prior  to 
task  execution,  or  not  present  at  all. 

The  most  effective  comparisons  to  make  for  these  categorizations  are  those  instances 
where  the  two  researchers  agreed;  for  most  dimensions  these  are  fewer  than  the  total  number 
of  incidents  studied  (N  =  16).  That  is,  the  number  of  incidents  used  in  the  agreement 
formula  between  researchers  and  BCs  represents  the  number  of  incidents  where  both 
researchers  agreed  about  the  presence  of  the  behavioral  dimensions  during  task  execution. 

For  each  dimension.  Table  4  presents  the  proportion  of  incidents  in  which  the  BCs  agreed 
with  researchers  about  the  presence  or  absence  of  a  dimension.  Note  that  only  one  BC  was 
involved  in  and  rated  each  incident.  Therefore,  these  data  compare  the  ratings  of  two 
researchers  with  one  BC  for  each  incident. 

Clearly,  there  is  a  high  level  of  agreement  among  the  researchers  and  the  Battalion 
Chiefs  for  some  dimensions,  particularly  those  regarding  the  four  dimensions  of  Team 
Resources,  and  the  dimensions  Roles  and  Functions,  Engaging,  and  Situation  Assessment. 
Agreement  on  two  of  the  dimensions  was  marginal;  Time  Horizon  (61.5%  agreement)  and 
Monitor  and  Adjust  (70%  agreement).  Agreement  on  the  remaining  four  dimensions  was 
poor;  Compensating  (30.8%),  Envisioning  Goals  (0%),  Managing  Uncertainty  (53.8%),  and 
Time  Management  (60%). 

The  most  reasonable  explanation  for  these  disagreements  over  the  presence  of 
Compensating,  Envisioning  Goals,  Managing  Uncertainty,  and  Time  Management  is  that 
there  was  not  an  understanding  of  the  dimensions  common  to  both  the  researchers  ^d  the 
BCs  or  that  the  definitions  were  applied  differently.  However,  one  interesting  finding  was 
that  of  the  37  total  disagreements  among  the  BCs  and  the  researchers,  the  BCs  indicated  that 
they  observed  a  dimension  when  the  researchers  did  not  during  29  of  the  incidents.  This 
finding  may  indicate  that  the  BCs,  experienced  in  the  domain,  noticed  some  cues  or 
behaviors  that  were  not  evident  to  the  researchers.  Several  examples  drawn  from  the 
incidents  indicate  where  this  may  have  occurred. 

Compensation  is  about  anticipating  the  needs  of  others  and  offering  assistance, 
anticipating  future  events  and  acting  to  meet  future  needs,  and  taking  opportunities  to  tram 
less-compentent  members  in  an  attempt  to  increase  their  ability  to  contribute  more  effectively 
to  the  team  goal.  There  are  two  resons  why  researchers  did  not  observe  compensation 
behaviors  when  Bcs  did.  First,  the  researchers  were  not  positioned  geographically  to  see  the 
behaviors,  and  second,  the  Bcs  noticed  subde  nuances  of  the  situation  which  the  researchers 
did  not  have  the  knowledge  to  notice. 
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Table  4 


Proportion  of  Agreement  Between  Battalion  Chiefs  and  Researchers 


Number  of 

% 

Behavioral  Dimension 

Cases 

Asreed 

Team  Resources 

Member  Resources 

16 

100 

Leader  Resources 

16 

100 

Non-human  Resources 

16 

87.5 

Procedures 

16 

87.5 

Team  Identity 

Defining  Roles  and  Functions 

16 

87.5 

Engaging 

14 

92.8 

Compensating 

13 

30.8 

Team  Coenition 

» 

Envisioning  Goals 

9 

0 

Assessing  the  Situation 

15 

100 

Focusing  on  Range  of  Factors 

9 

66.7 

Focusing  on  Time  Horizon 

13 

61.5 

Managing  Uncertainty 

13 

53.8 

Team  Self-Manaeement 

Monitoring  and  Adjusting 

10 

70.0 

Time  Management 

15 

60.0 
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When  a  ladder  company  put  up  a  28-foot  ladder  instead  of  a  35-foot  ladder  in  a  Police 
Assistance  response,  this  is  an  example  of  the  team  not  anticipating  the  needs  of  the  police. 
The  police,  who  are  not  as  skilled  with  climbing  ladders  as  firefighters  are,  can  get  onto  a 
roof  more  easily  with  a  longer  ladder.  The  BC  told  the  ladder  company  to  use  the  35-foot 
ladder  instead  and  explained  the  need  for  it.  The  researchers  did  not  see  this  interaction 
between  the  BC  and  the  ladder  company  officer.  Another  incident  which  was  missed  by 
researchers  but  is  clearly  a  behavior  which  indicates  compensation,  is  when  a  BC  described 
to  an  officer  the  procedures  for  handling  witnesses  to  a  firebombing. 

Firefighters  are  trained  to  anticipate  future  needs  and  to  be  flexible  in  their  roles. 

This  flexibility  occurs  when  changes  in  the  situation  are  communicated,  allowing  firefighters 
to  move  in  and  out  of  various  positions.  In  one  incident,  a  rescue  team  responded  to  a  fully- 
involved  fire  in  a  three-story  rowhouse.  They  immediately  began  helping  the  ladder 
company  vent  the  roof.  The  BC  in  command  of  this  fire  used  this  as  an  example  of 
compensation,  but  the  researchers  believed  the  rescue  team  was  simply  doing  their  assigned 
role.  It  is  difficult  for  an  outsider  to  know  what  is  inside  or  outside  of  the  realm  of  role 
responsibility. 

Some  companies  are  given  the  freedom  by  their  BCs  to  change  the  typical  response 
pattern  depending  upon  the  conditions  of  the  situation.  For  instance,  the  first-amving  engine 
company  positions  itself  at  the  front  of  a  building  and  the  second-arriving  engine  takes  the 
back.  In  one  incident,  the  first-arriving  engine  passed  around  the  back  of  a  building  where 
the  officer  noticed  a  fire  that  was  in  risk  of  spreading.  Instead  of  going  to  the  front  of  the 
building,  they  stopped  in  the  back.  In  this  case,  the  BC  observed  the  team  compensate  for 
what  was  a  restricted  SOP  for  this  incident. 

The  intricacies  of  a  domain  will  often  go  unnoticed  by  an  outside  observer,  leaving 
them  at  a  disadvantage.  Discussions  with  the  BCs  reinforced  our  belief  that  different 
instantiations  of  behaviors  wiU.  emerge  for  different  domains,  illuminating  the  importance  of 
using  SMEs  to  develop  a  team  performance  measurement  instrument. 

Time  management  was  also  observed  more  times  by  the  BCs  than  by  the  researchers. 
BCs  reported  the  need  to  manage  time  if  the  situation  has  the  potential  for  worsening  and  it 
is  necessary  to  take  specific  actions  to  circumvent  negative  outcomes.  Time  management 
occurs  as  part  of  the  BC’s  situation  assessment  and  this  may  be  the  reason  BCs  rated  its 
presence  more  often  than  the  researchers. 

BCs  observed  behaviors  representing  the  behavioral  dimension.  Envisioning  Goals  in 
each  of  the  16  incidents  whereas  the  researchers  observed  them  in  only  four.  BCs  indicat^ 
that  Envisioning  Goals  is  a  team  performance  dimension  that  is  considered  by  the  team  prior 
to  execution.  Many  fire  departments  conduct  informal  lessons-leamed  sessions  to 
discuss  how  they  might  have  fought  the  fire  differently.  This  gives  the  team  an  opportunity 
to  clarify  the  commander’s  goal  and  the  actions  to  reach  those  goals.  In  this  domam,  the 
companies  know  their  commander’s  goal  before  executing  the  task.  Researchers  did  not 
observe  this  dimension  during  task  execution. 


15 


The  dimension,  Envisioning  Goals,  is  a  critical  team  performance  process.  It  can 
become  a  problem  on  the  fireground  when  a  BC  expects  his  battalion  members  to  know  his 
intent  without  communicating  it  explicitly  at  any  point.  This  was  the  situation  in  one 
battalion  where  the  BC  ordered  a  company  to  the  roof  with  the  expectation  that  they  would 
begin  ventilation.  Instead,  the  company  went  to  the  roof  and  waited.  When  the  BC  noticed 
that  the  fire  did  not  do  what  he  expected  it  to  do  if  the  ventilation  had  occurred,  then  he 
radioed  the  officer  on  the  roof  to  ask  him  what  was  going  on.  The  officer  reported  that  he 
did  not  know  that  the  BC  wanted  him  to  vent.  The  BC’s  intent  was  for  firefighters  to  vent 
anytime  they  are  sent  to  the  roof,  unless  otherwise  directed. 

Behaviors  representing  the  behavioral  dimension  Managing  Uncertainty  were  observed 
more  by  the  BCs  than  by  the  researchers.  We  are  aware  of  three  possible  explainations  for 
this  discrepancy.  First,  the  behaviors  listed  by  the  BCs  as  indicative  of  this  dimension 
included  gathering  information  and  passing  it  onto  the  BC  who  then  develops  a  more  accurate 
and  certain  picture.  This  is  very  simitar  to  the  definition  of  Assessing  the  Situation.  It  is 
possible  that  researchers  did  not  convey  the  specific  differences  between  Managing 
Uncertainty  and  Assessing  the  Situation,  causing  the  BCs  to  use  the  behavioral  dimensions 
interchangeably.  The  second  reason  may  be  that  for  situations  lasting  less  then  30  minutes, 
the  act  of  assessing  the  situation  is  strictly  the  act  of  reducing  and  managing  uncertainty.  In 
longer  incidents  or  for  other  domains,  the  need  to  manage  uncertainty  may  be  greater.  And 
finally,  the  act  of  managing  uncertain  often  involves  recognizing  and  dealing  with  missing  or 
incomplete  information,  i.e.,  knowing  what  you  do  not  know.  Since  it  is  often  difficult  for 
outside  observers  to  see  behaviors  that  are  present,  it  is  not  hard  to  imagine  that  seeing 
behaviors  that  are  absent  would  be  even  more  difficult. 

Similar  to  the  researchers,  the  BCs  considered  the  performance  of  their  teams  as  very 
effective.  Table  5  presents  these  data  for  each  of  the  behavioral  dimensions.  Again  there 
was  a  ceiling  effect;  the  lowest  mean  rating  of  effectiveness  was  4.3  on  a  5-point  scale. 
However,  there  was  more  variability  in  the  BCs  ratings. 

We  designed  two  questionnaires  to  address  issues  concerning  useability  and 
acceptability.  The  first  questionnaire  addressed  the  issues  about  whether  or  not  this 
instrument  directed  the  BCs  attention  to  critical  team  performance  processes  and  if  it  helped 
the  BC  establish  training  recommendations. 

We  were  interested  in  the  BCs  subjective  rating  of  overall  team  performance  for  each 
of  the  18  incidents.  (Note:  The  data  from  the  two  incidents  not  used  in  the  previous 
analysis  was  considered  appropriate  to  use  here).  We  used  a  5-point  rating  scale,  with  a 
scale  value  of  one  anchored  to  ineffective  and  a  scale  value  of  five  anchored  to  highly 
effective.  BCs  used  a  rating  of  highly  effective  for  12  of  the  teams.  Of  the  six  teams  that 
received  a  lower  rating,  three  received  a  rating  of  4,  and  three  received  a  rating  of  3.  Two 
points  can  be  made  from  these  data.  Teams  whose  performance  was  rated  highly  effective 
exhibited  more  behaviors  indicative  of  the  ATDM  2.0  dimensions  than  teams  whose 
performance  was  rated  less  than  highly  effective.  In  most  cases,  if  the  team’s  overall 
performance  was  rated  highly  effective  their  performance  on  the  behavioral  dimensions  was 
also  rated  highly  effective. 
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Tables 


Mean  Effectiveness  Ratings  Provided  by  Battalion  Chiefs 


Behavioral  Dimension 

Mean 

S.D. 

N 

Member  Resources 

4.7 

.9 

12 

Leader  Resources 

4.7 

.5 

12 

Non-Human  Resources 

4.3 

1.2 

10 

Procedures 

5.0 

0 

10 

Defining  Roles  and  Functions 

4.8 

.4 

10 

Engaging 

4.8 

.6 

10 

Compensating 

4.5 

.8 

12 

Envisioning  Goals 

4.8 

.6 

12 

Focusing  on  Range  of  Factors 

4.6 

.9 

9 

Focusing  on  Time  Horizon 

5.0 

0 

5 

Managing  Uncertainty 

4.5 

.8 

10 

Assessing  the  Situation 

4.7 

.6 

12 

Monitoring  and  Adjusting 

4.4 

1.0 

9 

Time  Management 

5.0 

0 

6 

Our  next  area  of  interest  concerns  whether  the  instrument  directs  the  BCs’  attention  to 
areas  of  performance  where  intervention  is  most  needed.  To  answer  this  we  asked  BCs  to 
list  the  team  processes  they  would  like  to  see  improved  for  each  incident.  We  also  asked 
them  to  recommend  an  intervention  for  the  team  operating  during  each  incident.  For  six 
incidents  where  the  BCs  rated  the  teams  overall  performance  less  than  highly  effective, 
processes  and  interventions  were  identified  for  five  of  them.  For  the  other  12  incidents 
where  the  teams  were  rated  highly  effective  on  their  overall  performance,  only  two  processes 
were  identified  as  deficient.  Interventions  to  improve  the  teams’  performance  were  identified 
for  only  three  of  these  12  incidents.  For  instance,  one  firefighter  from  an  engine  company 
was  detailed  to  a  ladder  company.  The  BC  noted  that  he  did  not  perform  well  as  ladder 
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personnel,  that  he  broke  out  too  many  windows.  Two  recommendations  were  made  by  the 
BC.  One  was  to  provide  this  firefighter  specific  ladder  training  and  the  other  was  for  the  BC 
to  be  more  specific  with  his  instruction  in  these  kinds  of  situations.  To  summarize,  the 
instrument  focused  the  BCs  attention  on  the  team  performance  processes  that  are  in  need  of 
improvement  and  directed  the  BCs  to  identify  interventions  for  improving  these  processes. 
Whether  or  not  the  interventions  identified  by  the  BCs  would  improve  the  team’s 
performance  was  not  examined  in  this  study. 

Finally,  we  asked  the  BCs  to  rate  the  usefulness  of  the  instructional  package.  All  six 
of  the  BCs  reported  that  the  package  helped  them  to  understand  team  performance.  Five  of 
the  six  reported  that  the  package  helped  them  to  improve  team  performance  on  line.  The  one 
BC  who  disagreed  with  this  statement  reported  that  the  booklet  wih  not  help  team 
performance  on  the  fireground  because  team  performance  is  taught  to  firefighters  from  the 
first  day  on  the  job  and  every  day  there  after. 

Six  of  the  BCs  reported  that  the  instrument  would  help  them  conduct  After  Action 
Reviews  by  helping  them  to  identify  and  reinforce  what  went  well  and  what  did  not.  One 
BC  reported  the  criticality  of  analyzing  team  performance  to  improve  the  effectiveness  of 
post-incident  analysis.  Six  of  the  six  reported  the  instrument’s  usefulness  for  helping  them  to 
convey  team  performance  concepts  to  firefighters.  One  BC  reported  how  helping  the 
firefighters  understand  team  performance  concepts  will  help  the  incident  commander 
organize,  manage,  and  coordinate  fireground  operations. 

Finally,  only  three  of  the  BCs  thought  the  package  contained  appropriate  language  for 
firefighters.  Some  suggestions  include  wording  everything  in  fire-fighting  language, 
simplifying  terms  like  "Time  Horizon"  to  "time  to  accomplish  task,”  and  reducing  the  length 
of  the  instructions.  One  positive  part  of  the  instrument  concerns  the  listing  of  behaviors  to 
help  the  user  understand  the  meaning  of  the  behavioral  dimensions.  One  BC  commented 
positively  about  how  the  examples  of  behaviors  under  each  behavioral  dimension  helped  him 
to  understand  its  meaning. 

Findings  and  Conclusions 

The  measurement  instrument  based  on  the  ATDM  2.0  model  is  currently  a  draft 
version.  It  is  not  ifuUy  ready  for  use  in  field  settings.  However,  the  field  study  reported 
here  produced  several  interesting  findings  that  indicate  the  instrument  has  considerable 
promise  for  assessing  and  diagnosing  the  dynamic  processes  of  team  performance  in 
operational  environments.  The  following  paragraphs  describe  several  of  these  findings. 

The  first  concerns  the  utility  of  the  ATDM  2.0  model  for  describing  team 
performance  on  the  urban  fireground.  Because  the  origins  of  the  model  were  in  military 
command  and  control,  we  had  concerns  that  portions  of  the  model  would  not  be  appropriate 
for  firefighting  teams  or  that  elements  key  for  describing  fiirefighting  teams  would  have  been 
omitted.  We  found  that  this  was  not  the  case.  The  observers  and  domain  SMEs  noted 
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examples  of  each  of  the  behavioral  dimensions  as  they  evaluated  the  various  teams,  and  each 
of  these  dimensions  had  some  critical  value  during  at  least  one  of  the  incidents  observed. 

The  model  accurately  captured  the  team  processes  required  for  successful  performance  on  the 
urban  fireground. 

In  building  the  ATOM  2.0  model,  our  intent  was  to  describe  critical  team  processes  at 
a  general  level.  That  is,  the  model  was  to  be  a  model  of  team  processes  that  generalized 
across  a  large  array  of  team  domains.  The  levels  of  team  components  and  behavioral 
dimensions  allowed  this  generality.  Specificity  for  a  particular  domain  was  to  be  achieved  at 
the  lowest  level  of  the  model,  the  behavioral  markers.  The  markers  are  observable 
behaviors,  specific  to  the  domain,  that  indicate  the  quality  of  performance  on  a  certain 
behavioral  dimension. 

Our  concern  was  that  identifying  these  markers  would  be  prohibitively  difficult,  thus 
limiting  our  ability  to  modify  the  measurement  instrument  from  one  domain  to  another.  In 
fact,  adapting  the  draft  instrument  from  the  original  domain  (battle  staff)  to  urban  firefighting 
teams  proved  relatively  simple.  Many  of  the  markers  obtained  through  interviews  with 
Army  commanders  proved  useful  for  firefighting.  There  is  a  considerable  overlap  in  the  two 
domains,  particularly  regarding  the  command  structure.  Furthermore,  eliciting  markers  from 
the  Army  commanders  provided  us  a  template  and  prepared  us  to  do  the  same  with  Battalion 
Chiefs  (BCs).  With  relatively  few  interviews  (n=9),  we  were  able  to  build  a  comprehensive 
list  of  behaviors  that  represented  performance  on  each  of  the  dimensions  for  the  firefighting 
teams.  These  markers  were  then  converted  to  "indicators  of  effective  performance"  for  the 
measurement  instrument. 

A  second  set  of  findings  involve  the  effectiveness  of  the  instrument  itself.  We  found 
that  researchers  could  apply  the  measurement  with  a  high  degree  of  reliability  when  assessing 
the  performance  of  the  firefighting  team,  both  in  terms  of  identifying  the  relevant  dimensions 
and  rating  the  quality  of  team  performance.  Observers  who  were  extremely  familiar  with  the 
ATDM  2.0  model  and  with  team  performance  in  other  domains,  could  readily  recognize  and 
identify  key  behaviors  in  this  domain.  Interestingly,  the  domain  SMEs  also  showed  a  high 
degree  of  reliability  with  the  ratings  provided  by  the  researchers.  Thus,  the  researchers 
could  communicate  the  underlying  tenets  of  ATDM  2.0  and  the  BCs  could  recognize  critical 
behaviors  on  the  basis  of  this  model.  The  data  presented  here  did  identify  some  areas  of 
weakness  (e.g.,  envisioning  goals  and  time  management).  Prior  to  fielding  the  measurement 
instrument,  communication  and  understanding  of  these  areas  wiU  need  to  be  strengthened. 

The  firefighting  teams  investigated  in  the  field  study  were  aU  proficient,  high 
performance  teams.  Each  of  the  BCs  was  extremely  experienced  and  proficient  at  his  job. 
Similarly,  the  team  members  were  experienced  and  well  trained.  Most  of  the  teams  had 
worked  together  for  years.  This  produced  a  ceiling  effect  for  the  ratings  of  the  quality  of 
team  performance.  Therefore,  we  now  know  how  the  measurement  instrument  wih  work  for 
highly  proficient  teams,  but  not  for  less  proficient  teams.  Whether  the  measurement 
instrument  is  sensitive  to  changes  in  team  performance  remains  an  unanswered  question. 
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A  final  set  of  issues  concerns  the  BCs’  acceptance  of  ATDM  2.0  and  the 
measurement  instrument.  We  found  this  acceptance  to  be  high.  In  many  cases,  the  BCs 
found  the  concepts  in  ATDM  2.0  and  the  measurement  instrument  to  reflect  their  own 
thinking  about  team  performance  in  this  domain.  They  were  able  to  put  these  concepts  into 
the  language  of  the  firefighter.  Interviews  revealed  that  many  of  the  BCs  believed  they  had  a 
need  to  improve  their  team  training,  and  that  the  model  and  measurement  instrument 
provided  them  the  tools  to  focus  this  training  on  critical  team  processes.  However,  the 
useability  of  the  instrument  can  be  improved.  It’s  current  form  is  too  long  and  bulky.  BCs 
would  experience  difficulty  completing  the  instrument  during  the  course  of  an  incident.  It 
needs  to  be  abbreviated  and  constructed  in  a  way  to  make  it  conducive  for  use  and  reference 
in  the  field. 
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Appendix  A 

Battle  Staff  Team  Performance  Assessment  Tool  (TPAT) 
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TTT  AM  rHAH/^rTFRTSTICS.  This  component  refers  to  the  formation  or  foundation  of  a 
The  characteristics  may  represent  the  individuals  on  the  team  (e.g.,  competency  on 
individual  skills)  or  the  team  itself  (e.g.,  the  degree  to  which  the  team  employs  shared 
practices).  Each  dimension  included  under  this  component  of  team  performance  has  a  direct 
impact  on  the  team’s  capability  for  achieving  a  goal.  There  are  three  dimensions  of  this 
component.  Each  is  defined  in  the  following  section  and  accompanied  by  the  behaviors  that 
represent  high  and  low  performance  on  each  dimension. 

1.  Member/Leadership  Competence.  This  dimension  refers  to  the  ability  of  the 
individual  team  members  to  perform  their  specific  tasks.  This  includes  the  leader’s  ability  to 
perform  leadership  tasks.  The  team  cannot  function  unless  its  individuals  have  a  certain 
basic  level  of  competence.  If  the  general  ability  level  is  low,  then  the  team  will  not  be 
c^iable  of  developing  sophisticated  plans. 

fflGH  Staff  members  are  not  vague;  they  use  specific  language  when  talking  about  their 
areas  of  specialty. 

Staff  members  discuss  the  behaviors  they  expect  the  enemy  to  display. 

Staff  briefings  reflect  the  integration  of  essential/relevant  staff  areas. 

Staff  briefings  reflect  doctrinal  practices. 

Staff  briefings  reflect  knowledge  of  tactical  techniques  and  procedures. 

Staff  briefings  reflect  knowledge  of  terms. 

Staff  briefings  reflect  knowledge  of  graphics. 

Staff  members  incorporate  Decision  Support  Template,  Synchronization,  and 
Execution  matrices  into  the  operations  order. 

The  DST,  Synchronization,  and  Execution  matrices  are  formatted  correctly. 

The  DST,  Synchronization,  and  Execution  matrices  match  the  mission  and  the 
conditions  of  the  battlefield. 

Overlays  reflect  accurate  information. 

All  products  have  been  cross-checked  with  the  overlay. 

The  S2,  S3,  and  other  appropriate  command  and  control  maps  are  updated  as  often 
as  is  necessary  to  keep  the  staff  informed  of  the  battlefield  situation. 
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The  appropriate  C2  maps  reflect  common  information. 

The  XO  calls  staff  to  huddle  around  the  common  map. 

Staff  members  have  tactical  knowledge  about  principals  of  defense. 


LOW  Staff  members,  more  often  than  not,  do  not  provide  the  commander  informed  input 
for  decision  making. 

Staff  members  are  working  in  their  own  staff  areas  when  they  should  be  coordinating 
tasks. 

Commanding  Officer  is  forced  to  request  information  about  the  battlefield  because  the 
staff  members  who  are  receiving  the  information  do  not  pass  it  on. 

When  information  from  higher  headquarters  is  received  by  the  staff,  it  is  not  being 
passed  to  the  CO  when  it  can  be. 

2.  Shared  practices.  This  dimension  refers  to  the  degree  to  which  the  team  effectively 
implements  procedures.  The  skill  with  which  these  procedures  are  implemented  evolves  out 
of  the  experience  of  operating  together.  This  dimension  refers  to  the  relevance,  quantity , 
and  quality  of  practice  a  team  employs  to  become  proficient  with  both  stan^d  operating 
procedures  and  non-routine  tasks.  One  indication  of  weakness  in  this  area  is  that  members 
are  unable  to  handle  routine  tasks  that  should  be  performed  as  if  they  were  second  nature. 

An  indication  of  strength  in  this  area  is  that  the  team  not  only  handles  routine  tasks 
»  effectively,  but  also  deals  effectively  with  non-routine  tasks. 

HIGH  When  an  order  is  given  that  requires  the  activation  of  a  standard  operating 

procedures  (SOP)  like  Warning  Orders,  wargaming,  etc.  the  staff  responds 
rapidly  and  automatically. 

Updated  charts  (combat  power,  timeline,  etc.)  are  present  and  accessible. 

Each  staff  area  maintains  its  status  in  terms  of  resources,  location  of  resources,  and 
what  they  are  doing. 

The  SOPs  match  the  desired  task. 

Established  TOC  SOP  checklists  are  visible  in  the  TOC. 

When  the  staff  receives  an  infrequently  occurring  requirement,  the  right  people  are 
notified  and  come  together  in  a  coordinated  fashion. 
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LOW  Staff  is  not  engaging  in  standard  operating  or  other  common  procedures. 

Staff  members  are  working  in  their  own  areas  with  little  interaction  with  other  staff 
areas. 

Staff  officers  question  the  accuracy  of  charts. 

The  XO  or  the  S3  is  tpibng  the  staff  sections  what  to  do  more  than  50%  of  the  time. 

The  XO  or  S3  is  continually  reminding  the  staff  to  perform  various  tasks  that  should 
be  guided  by  SOP. 

Refuse  to  non-routine  staff  procedures  is  poor. 

3.  Command  climate.  This  dimension  refers  to  the  entire  team’s  attitude  toward 
accomplishing  its  mission  as  well  as  the  command’s  guiding  policies  and  procedures.  The 
command  climate  is  created  and  cultivated  initially  by  the  commander,  but  is  influenced  by 
the  different  personalities  of  the  team.  Command  climate  is  reflected  in  how  the  t^ 
approaches  its  mission  on  a  daily  basis.  A  positive  command  climate  is  one  in  which  the 
team  members  maintain  healthy  discipline  toward  the  task,  are  assertive  in  fulfilling  their 
duties  and  the  team’s  goal,  and  respect  each  other.  The  opposite  of  this  represents  a  negative 
command  climate. 

HIGH  Staff  members  persist  in  following  procedures  (checklists,  manuals)  even 

though  they  are  tired  and  are  under  a  high  degree  of  stress. 

When  staff  members  are  tired  and  under  a  high  degree  of  stress,  radio  transmissions 
are  crisp  and  clear. 

Individuals  are  not  complaining  about  how  tired  and  overworked  they  are. 

In  the  interest  of  carrying  out  the  intent  of  the  CO,  staff  members  are  persistent  in 

making  their  needs  known  to  higher  headquarters  and  other  organizations. 

! 

Individuals  voice  divergent  views. 

Although  there  are  divergent  views,  all  staff  members  agree  to  work  on  the  decided 
courses  of  action  even  if  they  oppose  the  chosen  course. 

Staff  members  are  doing  their  assigned  tasks  without  being  told  and  are  keeping  the 
CO/XO  and  other  staff  members  informed. 
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At  some  point  in  the  play,  the  commander  talks  to  subordinates  either  as  a  group  or 
individually  about  how  something  could  have  been  done  better. 

LOW  There  is  friction  between  staff  members. 

Staff  members  do  not  take  initiative. 

Little  information  is  being  passed  between  staff  areas. 

Initiative  is  quashed  by  the  XO. 

There  is  very  little  bantering  between  staff  members.  Instead,  they  appear  guarded 
and  suspicious. 

During  sta^  huddles,  only  the  XO  talks. 

When  a  staff  member  offers  a  differing  view,  the  XO  puts  the  person  down. 

The  staff  appears  divided  into  cliques. 

Subjective  measurements  of  command  climate; 

HIGH  Staff  members  are  disciplined 

Staff  members  are  respectful 

Staff  members  defend  perspectives  in  a  respectful  and  disciplined  manner. 

LOW  Staff  members  appear  afraid  to  make  a  mistake 
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TEAM  IDENTITY.  This  component  concerns  the  extent  to  which  team  members  conceive 
of  the  as  an  interdependent  unit,  and  operate  from  that  perspective  while  engaged  in  the 
task.  Decentering,  from  a  team  perspective,  is  the  process  through  which  people  think  of 
themselves  more  as  members  of  a  team  and  less  as  individuals  performing  jobs.  There  are 
three  dimensions  of  this  component.  Each  is  defined  in  the  following  section  and 
accompanied  by  the  behaviors  that  represent  high  and  low  performance  on  each  dimension. 

1.  Defining  Roles  and  Resources.  This  dimension  refers  to  the  extent  to  which  team 
members  understand  the  task  responsibilities,  expertise,  and  roles  of  every  other  member.  In 
addition,  this  refers  to  the  extent  to  which  team  members  understand  the  resources  required 
by  the  tiun  for  performing  its  fimctions  (radio,  CP  environment,  information  sources). 

HIGH  Staff  members  use  SOP  and  checklists  to  complete  performance  tasks. 

Staff  members  refer  to  and  use  the  Table  of  Operations  and  Equipment  (TO&E)  and 
other  manuals  relating  to  resource  allocation. 

When  a  specific  event  occurs,  or  when  specific  information  is  received,  the 
interactions  between  members  look  automatic  as  they  coordinate  the  tasks  that  were 
pre-assigned  to  them. 

When  a  non-routine  event  occurs,  the  XO  directs  the  activity. 

Staff  is  drilled  to  make  sure  they  have  experience. 

Workload  is  shared  by  all  members  of  a  staff  section. 

LOW  A  particular  task  was  not  completed  because  some  staff  member/s  dropped  the  ball 
during  a  staff  activity. 

Some  staff  members  do  not  perform  their  tasks  until  they  are  told  to  do  so. 

Staff  members  are  not  responding  to  orders  and  events. 

Only  one  person  is  completing  the  tasks  while  others  are  sitting  around. 

2.  Engaging  all  members.  This  dimension  refers  to  the  extent  to  which  the  team 
members  participate  in  the  team’s  work  and  take  responsibility  for  reaching  the  team  goals. 
This  dimension  includes  behaviors  that  signify  self  involvement  as  well  as  the  encouragement 
of  other  members  to  get  involved. 

HIGH  Chief  of  staff  actively  encourages  involvement. 

Staff  members  keep  the  CO  informed  of  the  status  of  the  battlefield. 
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Staff  members  keep  staff  leaders  informed  of  the  status  of  the  battlefield. 


Staff  members  keep  other  staff  members  informed  of  the  status  of  the  battlefield. 

Individuals  articulate  their  own  individual  experiences  with  particular  types  of 
operations. 

Both  competent  and  less-experienced  staff  members  are  fully  engaged  during 
activities  that  require  interaction. 

LOW  The  ^eadp^^^  asks  staff  members  for  information  about  status  and  progress. 

People  are  sitting  around  doing  nothing. 

A  majority  of  the  staff  is  sitting  in  their  own  areas,  and  don’t  come  out  except  when 
called. 

All  activity  is  closely  directed  by  staff  leaders  and/or  the  XO. 

The  S2  and  the  S3  don’t  have  the  same  thing  on  their  maps. 

3.  Compensating  and  coaching.  This  dimension  refers  to  the  extent  to  which  team 
resources  are  shifted  to  cover  areas  where  the  team  cannot  satisfy  its  roles,  functions,  and 
responsibilities.  Compensating  is  evidenced  when  team  members  step  appropriately  outside 
of  their  assigned  team  roles  or  functions  or  add  to  their  assigned  functions  in  order  to  help 
the  team  reach  its  goals.  Coaching  occurs  when  more  experienced  members  offer  direction 
to  less-experienced  members  for  the  purpose  of  bringing  individual  team  members  to  a 
greater  level  of  individual  proficiency,  thus  bringing  the  entire  team  to  a  more  advanced  level 

of  performance. 

HIGH  Senior  staff  are  working  with  junior  staff  in  an  attempt  to  cross  train  them. 


Leaders  are  coaching  subordinates  to  use  Manuals  and  templates  in  order  to 
understand  their  roles  and  the  battlefield  situation. 

TOC  shifts  are  balanced  in  terms  of  equal  capabilities  among  staff  members. 

More-experienced  staff  members  ask  questions  of  less-experienced  staff  members  in 
a  way  that  stimulates  the  less-experienced  member  to  think  about  what  they  are 
doing  and  to  get  them  to  consider  things  that  they  wouldn’t  have  considered  before. 
Members  who  have  experience  with  specific  operations  are  caUed  to  perform  a  role 
outside  of  their  normal  position. 

During  breaks  you  will  see  people  rehashing  what  they  have  just  finished  and 
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LOW 


hammering  out  ways  it  could  have  been  done  better.  In  a  high-performing  team  the 
squad  leader  may  or  may  not  initiate  this  discussion. 

Hi-trained  staff  members  are  compensated  for  in  the  short  term,  but  are  not  coached 
or  trained  in  that  task. 

Leaders  give  staff  members  the  answers  instead  of  referring  them  to  the  doctrinal 
manuals,  101-5  and  71-2. 

Staff  members  are  not  referring  to  doctrinal  and  other  manuals  that  will  allow  them 
to  become  more  e3q)erienced  and  knowledgeable  about  their  job. 

There  appears  to  be  Motion  among  team  members. 
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TF.AM  PI.ANNmsra  AND  DECISION  MAKING.  This  component  concerns  the  team’s 
ability  to  operate  as  an  intelligent  entity  that  thinks,  solves  problems,  makes  d^isions,  and 
actions  collectively  so  that  task  demands  are  met.  There  are  five  dimensions  of  this 
component.  Each  is  defined  in  the  foUowing  section  and  accompanied  by  the  behaviors  that 
represent  high  and  low  performance. 

1.  Envisioning  goals.  This  dimension  refers  to  the  extent  to  which  the  goals  of  the  team 
are  understood  and  used  to  direct  the  team’s  behavior  toward  those  goals.  There  are  three 
essential  aspects  to  envisioning  goals;  the  team’s  ability  to  identify  its  goals,  the  team’s 
ability  to  ensure  that  all  members  of  the  team  share  a  common  definition  of  the  goals,  and 
the  degree  to  which  this  shared  understanding  contains  a  workable  temporal  element.  In 
advanced  teams,  the  understanding  occurs  early  and  continues  throughout  the  exercise. 

The  most  critical  aspect  for  achieving  a  common  focus  is  for  the  team  to  know  what  the  CO 
has  in  his  mind. 

HIGH  CO  uses  standard  terms  and  graphics  contained  in  the  military  manuals. 

When  a  commander  uses  terminology  which  is  unique  to  the  unit,  he  describes  exactly 
what  is  meant. 

When  conveying  intent,  along  with  conveying  what  he  wants  to  happen,  the  CO  also 
refers  specifically  to  what  he  doesn’t  want  to  happen. 

Available  resources  are  addressed  and  considered  when  determining  goals  and  intent. 

Real-world  constraints  and  limitations  are  considered  when  determining  Commanders 
Intent  and  goals. 

Briefbacks  are  employed. 

Cl  is  repeated  in  multiple  documents,  briefings,  etc. 

2.  Maintaiiiing  dynamic  focus:  This  dimension  refers  to  the  team’s  ability  to  focus  its 
planning  and  decision  making  within  an  appropriate  span  of  time  (time  horizon)  and  on  a 
relevant  breadth  of  concepts  and  information  (range  of  factors).  It  concerns  the  team’s 
ability  to  focus  its  attentional  capacity  on  the  most  relevant  cues.  The  two  indicators  for 
adeqiiate  focus  are: 

Time  horizon.  There  are  three  aspects  of  time  horizon. 

(1)  Awareness  of  the  time  it  takes  for  an  action  to  be  accomplished  given  the 

constraints  of  the  environment. 
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(2)  Awareness  of  multiple  time  horizons  during  planning  and  execution. 

(3)  The  team’s  ability  to  focus  ahead  to  anticipate  and  act  on  upcoming  events. 

TnOH  Timelines  are  developed  that  include  times  from  mission  receipt  to  mission 

conclusion. 

Timelines  include  the  involvement  of  successive  echelons. 

Subordinate  elements  of  the  force  develop  timelines. 

It  is  clear  which  military  functions  are  and  are  not  included  in  the  timelines. 

The  timeline  produced  by  the  XO  matches  the  staffs  timeline. 

Staff  uses  a  planning  factors  workbook  which  contains  realistic  times  for  battlefield 
occurrences. 

Lower  echelons  receive  orders  in  enough  time  to  implement  them. 

The  timeline  was  realistic. 

Products  such  as  situation  template  for  portrayal  of  enemy  in  the  planning  phase, 
synchronization  matrices,  and  operation  overlays  are  finalized  and  distributed  in 
enough  time  to  be  used  as  input  to  another  product. 

Staff  areas  proactively  seek  information  which  will  assist  future  planning  and 
activities. 

Staff  areas  think  through  batde  activities  and  develop  contingency  plans. 

LOW  Future  events  are  anticipated,  but  no  action  is  taken  to  set  up  a  response  to  the  event. 

3.  Range  of  factors.  This  indicator  concerns  the  team’s  awareness  of  the  many  factors 
that  are  considered  for  planning  and  decision  making.  The  concept  of  range  of  factors  refers 
to  the  team’s  ability  to  consider  an  appropriate  number  of  factors;  not  so  many  that  it 
becomes  paralyzed,  and  not  so  few  that  the  solution  results  in  error. 

HIGH  Relevant  range  of  factors  is  identified  during  planning. 

Relevant  range  of  factors  is  considered  during  situation  assessment. 

Relevant  range  of  factors  is  identified  during  decision  making. 

Staff  assesses  the  mission. 
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Staff  areas  gather  information  and  facts  relevant  to  their  specific  areas  and  relevant  to 
specific  staff  huddles. 

Critical  factors  are  identified  and  prioritized  through  developing  the  Commander’s 
Critical  Information  Requirements  (CCIR),  Priority  Intelligence  Requirements  (PIR), 
and  Intelligence  Requirements  (IR). 

All  logs  are  updated. 

Adjacent  units  are  on  maps  and  overlays,  and  staff  are  looking  at  the  broad  context  of 
operations,  that  is,  beyond  the  confines  of  the  boundary  lines  of  other  units. 

Information  is  exchanged  with  adjacent  units,  maps  are  updated,  and  information  is 
disseminated  as  a  result  of  this  exchange. 

The  context  of  the  battle  is  discussed  during  shift  changes. 

When  a  METT-T  change  occurs,  the  context  of  the  battle  is  addressed. 

Complete  range  of  issues  are  identified  and  articulated  to  the  CO  and  the  XO  during 
briefings  and  staff  huddles,  exchanged  between  relevant  staff  members,  and  logged 
and  posted. 

Critical  issues  (as  opposed  to  factors)  are  identified  and  prioritized. 

Staff  is  coordinating  and  interacting  with  higher  headquarters. 

*  LOW  The  operational  issues  that  must  be  addressed  by  the  CO/Staff  are  not  identified. 

All  potential  combat  power  is  not  considered. 

Staff  operational  charts  and  reference  materials  are  not  used. 

Records  of  what  has  and  has  not  been  done  are  not  kept. 

Activities  outside  the  immediate  sector  or  area  of  responsibility  are  not  considered. 

During  shift  changes  and  changes  in  METT-T,  the  CO  and  staff  lose  their  focus. 

The  CO  and  staff  do  not  focus  the  BDE/TF  on  the  scope  of  operations  throughout 
plan,  preparation,  and  execution. 

4.  Situation  assessment.  This  dimension  concerns  the  team’s  ability  to  form  and 
maintain  a  shared  understanding  of  the  situation  with  which  it  is  operating.  Arriving  at  a 
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shared  understanding  of  a  situation  involves  forming  plausible  scaiarios  about  what  is 
h^jpening  and  holding  onto  altanative  hypotheses  until  ambiguity  is  resolved.  In  a  team 
setting  where  more  than  one  assessment  of  the  same  situation  may  be  made,  the  processes  of 
diverging  and  converging  are  critical  for  bringing  the  team  to  a  common  and  rich 
understanding.  In  dynamic  environments  where  information  coming  into  the  team  can 
change  the  team’s  assessment  of  the  situation,  maintaining  a  common  understanding  is 
critic^. 

Divergence.  This  process  concerns  the  team’s  ability  to  seek  views  about  the 
situation  from  a  variety  of  team  members  and  to  hold  on  to  ambiguous  information 
until  it  can  be  resolved. 

TTTf:TT  Staff  members  are  attentive  and  interactive  (engaged)  during  issuance  of  CO’s 
planning  guidance. 

Staff  members  from  a  specific  area  of  expertise  comment  and  provide  input  to  their 
own  critical  mission  analysis  and  contribute  to  the  development  of  analyses  conducted 
by  other  staff  areas. 

Staff  members  use  sound  logic  when  arguing  their  positions. 

Staff  members  listen  and  interact  while  others  state  their  ideas/positions. 

Staff  members  seek  all  pertinent  factors  that  impact  their  viewpoints  or  others’ 
viewpoints. 

During  wargaming,  staff  members  articulate  various  options  about  "how  to  fight." 

Staff  members  understand  when  divergence  is  needed  and  positive. 

The  devil’s  advocacy  role  is  played. 

Different  interpretations  of  data  are  elicited. 

LOW  Staff  members  never  express  strong  positions. 

Staff  members  accept  the  "group"  view  without  question. 

5.  Convergence.  This  process  concerns  the  team’s  ability  to  harness  various  views  to 
form  a  situation  assessment  that  is  as  close  as  possible  to  ground  truth.  The  methods  used  to 
harness  the  many  and  varied  views  include  the  use  of  maps,  diagrams,  messages,  and 
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meetings  to  ensure  that  relevant  team  members  are  viewing  the  situation  similarly  and  are 
working  from  a  similar  perspective. 

high  The  ideas  or  concepts  embodied  in  the  operations  plan  are  documented  and 

someone  on  the  staff  team  announces  that  the  document  represents  what  was 
agreed  upon. 

Convergence  occurred  on  time. 

Once  a  decision  is  made,  the  entire  staff  goes  with  it. 

The  total  force  situation  is  discussed  during  staff  huddle. 

Information  about  the  decision  is  shared  among  staff  sections. 

The  command  net  is  set  up  with  all  necessary  function  "leaders"  on  it. 

Maps,  overlays,  charts,  staff  huddles,  and  briefings  are  used  to  convey  battlefield 
context,  display  and  verify  the  decision  made. 

When  the  announcement  is  made  that  the  documentation  reflects  everyone’s 
understanding  of  the  situation,  backbriefs  are  employed  to  ensure  common 
understanding, 

LOW  Staff  members  do  not  attempt  to  resolve  ambiguous  information. 

Only  one  individual  is  permitted  to  express  opinions. 

Staff  is  tracking  batde  preparation  and  battle  execution,  but  is  not  anticipating 
retjuired  involvement  nor  coordinating  information  with  the  rest  of  staff. 

6.  Maintaining.  A  situation  assessment  is  initially  developed,  but  as  the  battle  staff  take 
various  actions,  more  information  about  the  situation  is  gained,  thus  changing  the  assessment. 
Awareness  of  the  situation  must  be  maintained  throughout  each  stage  of  the  battle.  This 
involves  not  only  refining  the  situation,  but  also  ensuring  that  all  staff  members  have  a 
common  understanding  of  the  refined  assessment. 

HIGH  Maps,  overlays,  charts,  staff  huddles,  and  briefings  are  used  to  convey 

battlefield  context,  display  and  verify  the  decision  made  as  more  information 
comes  in. 

Staff  analyzes  new  input  to  determine  impact  on  current  situation  assessment  and 
talfp-s  action  to  coordinate  with  others. 
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Staff  huddles  are  called  to  summarize  perceived  ground  truth. 

The  situation  is  reflected  in  the  Ops  map. 

The  situation  is  reflected  in  the  Intel  map. 

Recent,  current,  and  projected  activities  of  the  friendly  and  enemy  forces  are 
discussed  in  the  staff  huddles. 

During  the  staff  huddles,  there  is  not  a  lot  of  disagreement  about  the  SA. 

When  information  comes  in  that  causes  reassessment  of  the  situation,  the  staff  goes 
through  the  cycle  of  diverging  and  converging. 

Staff  members  are  updating  information  through  each  phase  of  the  battle. 

Clear  logs  are  k:q)t  and  posted. 

LOW  Staff  is  not  cognizant  of  progress  of  preparation  in  their  area  of  responsibility. 

7.  Envisioning  and  evaluating  courses  of  action.  This  dimension  concerns  the  team’s 
ability  to  use  its  collective  experience  to  visualize  how  a  plan  will  be  coordinated  and 
conducted,  and  to  visualize  where  it  will  run  into  trouble. 

HIGH  Enemy  situation  templates  are  adapted  to  reflect  ground  truth. 

Discussion  among  members  indicates  they  have  visualized  the  map  terrain  as  3D,  with 
trees,  hills,  etc. 

Discussion  among  members  indicates  they  understand  the  language  and  terminology  of 
the  graphics. 

Staff  is  disciplined,  yet  flexible  (creative,  innovative,  imaginative)  while  wargaming 
the  COA. 

During  situation  assessment,  staff  members  think  through  alternative  courses  of 
action. 

All  available  friendly  and  enemy  combat  power  is  evaluated  during  wargaming  and 
each  staff  officer  provides  his  vision  of  the  impact  of  their  area  of  responsibility  on 
the  operation. 

Places  where  timing  might  be  tricky  or  difficult  are  identified  and  articulated  to  the 
staff,  the  CO,  and  the  XO  in  the  context  of  the  wargaming  process. 
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Staff  members  do  not  consider  the  effect  of  alternative  COAs. 

Staff  members  do  not  participate  in  "mentally"  fighting  the  battle. 

Staff  members  do  not  consider  tactical  imperatives. 

Staff  members  take  a  cookie  cutter  approach  to  visualizing  how  a  plan  might  develop. 

Staff  members  are  taking  a  rote  approach  to  applying  procedures. 

Staff  members  do  not  take  into  account  how  long  it  takes  for  certain  units  to  move  x 
distance. 

Staff  meetings  have  no  agenda;  people  are  bringing  up  subjects  that  are  not  relevant. 

Staff  members  do  not  conduct  evaluation  of  COA. 

8.  Articulating  expectations.  This  dimension  refers  to  the  extent  to  which  the  team 
articulates  its  expectations  about  the  progress  of  a  course  of  action  to  relevant  team  members. 
Formalizing  the  process  of  making  expectations  explicit  helps  the  team  attend  to  relevant 
information  and  serves  as  an  alerting  function,  thus  increasing  the  team’s  ability  to  grasp  the 
significance  of  a  message  and  react  quickly. 

HIGH  Expectations  are  based  on  where  the  enemy  is  and  where  it  is  not. 

S2  takes  the  Intelligence  Planning  of  the  Battlefield,  the  Situation  Template,  the 
terrain  and  weather  analysis,  and  joins  with  the  rest  of  the  staff  to  generate  the 
Decision  Support  Template  (DST). 

As  staff  members  analyze  their  expectancies,  there  will  be  divergent  views. 

Information  about  expectancies  is  given  to  the  staff  who  are  developing  the  DST. 

Executive  Officer/Chief  of  Staff  conducts  necessary  staff  "huddles"  to  bring  all  staff 
members  up  to  date  on  concept  development  and  situation  assessment. 

LOW  Contingency  options  are  not  identified. 

Staff  is  not  tracking  battle  progress. 

CDR/staff  does  not  articulate  his  expectations  relative  to  tactical  commitments  or 
employments. 
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TEAM  ST^T.F  MANAGFMFNT.  This  component  refers  to  the  team’s  ability  to  observe  its 
team  performance  processes,  recognize  its  level  on  the  team  characteristics,  and  make 
adjustments  to  reach  a  higher  level  of  performance.  There  are  four  dimensions  of  team  self 
managpmfMit  Each  is  defined  in  the  following  section  and  is  accompanied  by  the  behaviors 
that  represent  high  and  low  performance. 

1.  Monitoring.  This  dimension  refers  to  the  team’s  ability  to  examine  itself  for  signs  of 
both  effective  and  ineffective  teamwork  behaviors.  The  puipose  of  monitoring  is  for  the 
team  to  recognize  how  it’s  operating  as  a  team  and  make  adjustments  for  meeting  the  team 
task.  Recognizing  effective  behaviors  increases  the  team’s  ability  to  use  them  in  the  future. 

In  addition  to  monitoring  in  the  present,  the  team  can  look  into  the  future  to  anticipate  events 
that  win  require  specific  team  paformance  strategies. 

HIGH  Friction  between  staff  members  is  addressed. 

When  staff  members  recognize  that  they  are  falling  behind  the  timeline,  they  become 
more  focused  to  meet  the  outcome  on  time. 

Someone  on  the  staff  is  asking  staff  members  to  get  involved  in  the  task. 

If  staff  officers  are  not  bringing  their  area  of  expertise  into  the  wargaming  session, 
someone  is  making  sure  they  are. 

When  staff  members  are  not  functioning  normally  (because  of  being  ill  or  fatigued) 
someone  is  asking  about  their  welfare. 

Subordinate  staff  leaders  monitor  their  staff  sections  for  individual  performance, 
welfare,  and  attitudes. 

The  team  is  monitoring  all  dimensions  under  all  components  of  this  team  performance 
model. 

The  CO  stated  the  importance  of  team  performance. 

During  AAR,  team  performance  is  addressed. 

Members  of  staff  "police"  their  section  for  people  who  have  disengaged. 

Staff  shifts  are  monitored  to  remain  balanced  and  able  to  perform  necessary  tasks. 

LOW  Friction  among  team  members  is  outward  and  obvious,  with  some  members  angry 
with  one  another  and  unable  to  work  together. 

The  team  tasks  are  not  completed  on  time. 

During  AAR,  only  what  the  team  did  wrong  is  addressed. 
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2.  Adliusting.  This  dimension  refers  to  the  team’s  ability  to  change  team  performance, 
or  make  proper  adjustments  to  the  current  strategy  when  it  becomes  less  effective,  or  isn’t 
working.  It  also  refers  to  the  team’s  ability  to  anticipate  changes  that  may  require  different 
strategies  for  meeting  the  demands  of  a  future  task  or  situation. 

HTCT  Appropriate  changes  are  made  to  the  concept  of  operations  when  something  is 
not  going  as  expected  or  planned. 

Members  are  encouraged  to  use  initiative  when  operational  changes  are  required. 

When  necessary,  the  CO  modifies  existing  roles  and  functions  through  clear,  crisp 
direction. 

If  members  step  out  of  doctrinal  roles  and  functions,  they  do  so  according  to 
established  procedure. 

Staff  members  are  making  appropriate  decisions  at  their  level  of  authority  to  adjust 
previously  agreed  upon  plans. 

Adjustments  are  made  in  a  timely  manner. 

Staff  members  are  actively  recommending  changes  to  OPORDER  when  necessary. 

Staff  members  change  SOPs  when  the  previous  ones  fail  to  meet  the  goals  of  the  task. 

LOW  Staff  continues  to  support  initial  plan  even  though  battlefield  situation  has  changed. 

Staff  members  don’t  take  action  on  reports  that  indicate  action  is  needed. 

3.  Detecting  gaps  and  inconsistencies.  This  dimension  refers  to  the  extent  to  which  the 
team  actively  attempts  to  discover  and  fill  gaps  in  the  team’s  information  base  and 
assumptions;  to  recognize  and  handle  inconsistencies  or  contradictions  that  might  be  present 
either  through  eliminating  the  inconsistencies  or  noting  them  until  they  can  be  resolved. 

HIGH  The  CO  steps  away  from  the  activity  of  the  battle  staff  and  takes  time  to 

reflect  about  the  bigger  picture. 

Staff  members  ask,  "What  haven’t  we  considered?" 

Staff  members  ask,  "What  are  we  missing?" 

Staff  members  ask,  "What  don’t  we  know?" 

The  staff  is  using  briefbacks,  rehearsals,  and  staff  "huddles"  to  identify 
inconsistencies  and  gaps. 


A-17 


When  gaps  are  detected,  corrective  actions  are  recommended. 

Gaps  and  inconsistencies  are  brought  to  the  attention  of  appropriate  members  or  the 
COS. 

The  CO  identifies  gaps  and  inconsistencies  at  the  level  of  his  intent  statement. 

Staff  is  maintaining  situation  awareness. 

The  !gtaff  is  assessing  the  impact  of  gaps  on  the  overall  plan  and  corrective  actions  are 
recommended. 

LOW  The  whole  team  is  focused  on  one  COA  and  doesn’t  consider  contingency 
requirements. 

Details  are  missed  or  not  discovered  until  the  OPORDER  is  ready. 

Staff  does  not  properly  track  battle  preparation  and  does  not  assess  shortfalls  in  their 
area  of  responsibility. 

4.  Time  wianagpmpnt.  This  dimension  refers  to  the  team’s  ability  to  meet  goals  before 
dkdlines  overtake  them.  Managing  time  includes  sequencing  sub-tasks  so  that  output  from 
one  task  connects  where  and  when  it  should  as  input  to  the  next  task. 

HIGH  Staff  sets  up  timelines,  with  beginning  and  end  times,  that  trigger  staff  actions. 

Staff  sets  up  timeline  which  includes  all  necessary  process  steps. 

The  staff  engages  in  a  technique  called  "backward  planning." 

Staff  members  extract  LD  times,  back  up  2/3,  and  designate  the  time  that  the  order 
needs  to  be  completed. 

Staff  members  recognize  and  articulate  time  constraints  that  dictate  when  they  have  to 
"move  on." 

Time  Management  charts  are  completed  and  displayed  for  all  team  members. 

Time  Management  charts  contain  procedures  and  unit  operational  events  for  aU 
aspects  of  the  mission. 

Time  Management  charts  contain  timing  for  critical  and  complex  sub-missions. 

CO  monitors  the  master  time  management  chart. 

CO  and  staff  team  are  addressing  future  operations  appropriately. 
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Appendix  B 


Definitions 


and  Descriptions  of  Behavioral  Dimensions  for  Fireground 
Battalion  Chiefs 
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CATEGORY;  TEAM  RESOURCES 
Dimension:  Member  resources 

This  dimension  refers  to  the  team’s  understanding  of  its  members’  capabilities.  Capabilities 
inriudp-  the  sldlls,  knowledge,  mental  and  physical  abilities,  experiences,  and  personalities. 
What  is  critical  to  team  performance  is  not  so  much  knowing  how  "good"  the  individuals 
are,  but  knowing  what  resources  the  team  has  to  work  with.  By  having  a  good 
understanding  of  the  membCT  resources,  the  team  will  be  better  able  to  match  these  resources 

to  the  demands  of  the  task. 

Planning.  When  planning  who  will  do  a  particular  task,  the  team  should  ass«s  members’ 
strengths  and  weaknesses  and  make  sure  all  members  understand  what  is  available.  All 
members  should  know  who  brings  what  strengths. 

Execution.  Before  executing  a  task,  the  team  should  try  to  gain  an  understanding  of  its 
member  resources.  You’U  see  this  dimension  during  task  execution  if  members  are 
monitoring  the  on-going  status  of  team  members  and  making  appropriate  adjustments  in  the 
event  of  fatigue,  injuries,  or  inability  to  do  the  job. 

Indicators  of  effective  performance  in  the  firefighting  domain 

•  BC  ascertains  who  is  in  each  company  during  a  given  shift  prior  to  an  incident. 

•  BC  ascertains  the  competence  level  of  individuals  in  the  battalion. 

•  BC  listens  to  the  radio  and  knows  which  companies  are  responding  to  the  incident. 

•  BC  ascertains  who  can  and  who  cannot  handle  an  incident.  This  includes  the  ladder 
and  engine  companies,  the  medics,  and  the  officers. 

The  value  of  this  dimension  to  team  performance.  One  Battalion  Chief  was  called  to  a  fire 
across  town.  On  his  way  there,  the  engine  company  officer  placed  it  under  control.  The 
Battalion  Chief  believed  the  officer  put  it  under  control  because  he  knew  it  was  a  long  way 
for  the  Battalion  Chief  to  go;  he  was  not  yet  completely  confident  in  this  officer’s  skills. 

The  Battalion  Chief  ended  up  going  anyway  and  spent  an  hour  on  the  fireground.  The 
Battalion  Chief  s  knowledge  of  the  way  this  officer  worked  increased  the  operating  efficiency 
of  the  team. 

Methods  ip^ming  about  team  members.  The  capabilities  of  members  can  be  assessed 
through  observations,  informal  discussions,  interviews,  education  records,  and  resumes.  You 
may  also  employ  more  formal  assessment  methods  such  as  psychometric,  intelligence,  or 

skills  tests. 
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CATEGORY:  TEAM  RESOURCES 

Dimension:  Leadership  resources 


This  dimension  is  about  team  members’  knowledge  of  their  leaders’  capabilities  and 
preferences. 

Planning.  The  team  should  discuss  the  strengths  and  limitations  in  its  leadership  resources 
and  be  clear  about  its  leadership  needs. 

Execution.  Before  executing  a  task  the  team  should  have  knowledge  of  who  the  leaders  are 
and  an  understanding  of  how  they  will  operate.  If  modifications  to  leadership  are  made 
during  execution,  the  changes  need  to  be  communicated  to  the  whole  team. 

TnHirators  of  effective  performance 

•  BC  understands  what  the  officers  in  each  company  can  and  cannot  handle. 

•  BC  determines  who  is  in  each  company  during  a  given  shift. 

•  If  the  officer  of  the  ladder  or  engine  company  arrives  on  the  scene  before  the  BC,  all 
are  clear  about  who  has  leadership  responsibility  until  the  BC  arrives. 

•  When  the  BC  arrives,  he  receives  viable  information  and  then  takes  command. 

•  When  the  Deputy  Chief  arrives,  he  slowly  eases  himself  into  the  command  position. 
Taking  over  immediately  may  disrupt  execution  of  the  firefight. 

The  value  of  this  dimension  to  team  performance.  This  dimension  is  not  about  the  strength 
.  or  the  style  of  the  leader.  Instead,  it  is  about  the  team’s  knowledge  of  the  leader’s  sttength 
and  style.  Knowing  this  allows  the  team  to  work  within  a  particular  style.  Leadership 
functions  must  be  performed.  If  one  leader  is  incapable,  unwilling,  or  too  busy  to  fulfill  the 
functions,  then  these  functions  must  be  distributed. 

Methods  for  learning  about  leadership.  Leadership  resources  can  be  assessed  through 
observation  and  discussion,  or  through  formal  methods  such  as  leadership  profiles  and 
interviews.  Firefighters  should  be  encouraged  to  get  to  know  their  leaders. 
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CATEGORY:  TEAM  RESOURCES 

Dimension:  Non-human  resources 

This  dimension  refers  to  the  equipment,  appnrstus,  tools,  supplies,  vehicles,  and  computers/ 
software  that  are  available  to  the  team  to  get  the  job  done.  The  key  to  this  dimension  is  the 
knowledge  members  have  about  what’s  available,  its  capabilities,  and  how  it  operates. 

Planning.  When  making  plans  to  execute  a  task,  the  team  should  verify  what  resources  are 
available  to  them,  and  identify  weaknesses  and  strengths.  This  information  is  communicated 
to  the  whole  team.  If  resources  are  added,  then  the  team  needs  to  know  about  them. 

Execution.  The  team’s  understanding  of  the  resources  available  to  them  should  be 
ascertained  prior  to  task  execution.  You  will  see  this  dimension  when  changes  to  resources 
occur  during  execution.  These  changes  need  to  be  communicated  to  the  whole  team. 

Indicators  of  effective  performance 

•  BC  checks  the  dispatch  notification  sheet  to  find  out  what  equipment  he  will  have 
available  to  work  with  on  the  scene.  This  is  referred  to  as  the  company  randown. 

•  Each  team  member  knows  what  equipment  will  be  arriving  at  a  given  incident. 

•  Each  team  member  knows  which  resources  are  carried  in  the  various  vehicles. 

The,  value  of  this  dimension  to  team  performance.  Knowing  the  resources  that  are  available 
to  the  team  allows  members  to  take  steps  to  maximize  their  use.  Conversely,  knowing  what 
.  it  doesn’t  have  to  work  with  allows  the  team  to  work  around  the  absence.  For  example,  if 
Rescue  One  is  not  available  to  respond  to  a  job,  then  the  plan  of  action  must  include 
resources  to  cover  the  work  usually  performed  by  Rescue  One. 
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CATEGORY;  TEAM  RESOURCES 

Dimension:  Basic  Procedures 


This  dimension  refers  to  the  standard  procedures  that  are  in  place  to  provide  basic  direction 
and  processes  for  the  team  members.  Procedures  are  developed  to  guide  the  team  to  more 
efficient  and  effective  operation.  Procedures  are  enacted  by  team  members  through  a  series 
of  coordinated  activities.  Team  members  must  be  aware  of  these  procedures  and  how  they 
are  implemented.  The  smoothness  with  which  procedures  are  implemented  indicates  the 
team’s  familiarity  with  its  basic  procedures.  Familiarity  is  gained  through  both  study  and 
practice. 

Planning.  The  team  may  rely  on  procedures  during  planning  sessions  to  direct  their 
activities.  Developing  procedures  that  will  be  used  during  task  execution  is  different  from 
implementing  the  procedures.  Basic  procedures  should  be  established  m  advance  of  task 
execution  and  task  execution  will  uncover  needed  changes  or  additions  to  standard 
procedures.  During  the  planning  stage  we  should  see  the  team  verify  these  procedures, 
communicate  them  to  the  team  members,  and  incorporate  procedures  into  the  planning 
process. 

Execution.  Characteristics  of  the  situation  will  inform  the  team  about  which  procedure 
should  be  implemented.  The  characteristics  and  the  subsequent  procedure  are  established 
prior  to  task  execution.  During  task  execution  a  well-informed  observer  can  watch  to  see  if 
team  members  are  implementing  procedures  appropriately.  If  a  procedure  is  no  longer 
appropriate  for  the  characteristics  of  the  task  and  it  is  altered  or  discarded,  then  these 
changes  must  be  communicated  to  the  whole  team. 

Indicators  of  effective  performance 

•  When  the  team  responded  to  an  Assist  Police  caU,  all  sirens  were  cut  four  blocks 
away  from  the  scene. 

•  When  a  Tactical  Box  call  went  out,  two  engines,  two  ladders,  and  one  BC  responded. 

•  When  a  Full  Box  alarm  went  out,  four  engines,  two  ladders,  and  two  BCs  responded. 

•  The  first  arriving  engine  took  the  front,  the  second  engine  took  the  rear,  the  third 
engine  took  the  front,  and  the  fourth  engine  took  the  rear.  The  first  arriving  ladder 
took  the  front,  the  second  ladder  took  the  rear.  The  first  arriving  BC  took  the  front, 
the  second  arriving  BC  took  the  rear. 

•  When  nothing  was  showing,  the  first  arriving  BC  called  for  reduced  speed  of 
incoming  engines. 

•  The  first  arriving  officer  assumed  leadership  responsibility  until  a  higher  ranking 
officer  arrived. 
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CATEGORY:  TEAM  IDENTITY 
Dimension:  Defining  roles  and  functions 

This  behavioral  dimension  refers  to  the  articulation  and  understanding  of  the  task 
responsibilities,  functions,  and  roles  of  every  other  member.  Once  roles  and  functions  are 
understood,  the  team  can  take  advantage  of  the  individual  power  available  to  it.  Use  of  this 
behavioral  dimension  will  facilitate  coordination  efforts. 

Planning.  You  can  observe  this  dimension  during  planning  tasks  by  watching  to  see  if  team 
members  are  clarifying  their  roles  and  functions  and  making  sure  that  everyone  understands 
them.  Roles  and  fonctions  are  made  explicit. 

Fyecution.  Much  of  the  team’s  understanding  of  their  roles  and  functions  should  be  achieved 
prior  to  task  execution.  If  the  team  is  required  to  make  changes  in  roles  and  functions  on 
line,  then  these  changes  must  be  communicated  to  the  whole  team. 

Indicators  of  effective  performance 

•  When  alarms  greater  than  a  tactical  box  are  called,  the  BCs  discuss  the  roles  they  wiU 
take. 

•  BCs  reminded  firefighters  about  their  role  in  an  assist  police  call.  That  is, 
firefighters  provide  the  means  for  the  police  to  do  their  job  and  do  not  get  involved  in 
the  incident.  This  prevents  firefighters  from  getting  near  evidence  and  from  getting 
into  situations  they  are  not  quahfied  to  handle. 

•  Medics  let  firefighters  know  what  functions  they  can  serve  in  a  car  accident  situation. 

•  It  is  well  known  that  the  BC  Aide  will  serve  as  a  time  keeper  to  keep  track  of  how 
long  people  have  been  inside  a  structure. 
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CATEGORY:  TEAM  IDENTITY 
Dimension:  Engaging 

This  behavioral  dimension  represents  the  extent  to  which  the  team  members  participate  and 
encourage  others  to  participate  in  the  team’s  work  and  take  responsibility  for  reaching  the 
team  goals. 

Planning  and  Execution  behaviors  look  similar. 

Indicators  of  effective  performance 

•  When  an  officer  is  handling  a  situation,  the  BC  does  not  take  over  the  situation  unless 
it  is  necessary.  If  he  does  take  over,  it  may  discourage  the  officer  from  taking 
responsibility. 
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CATEGORY:  TEAM  IDENTITY 

Dimension:  Compensating 

This  dimension  refers  to  the  movement  of  team  resources  to  cover  areas  where  the  team 
cannot  satisfy  its  roles,  functions,  and  responsibiUties.  Compensation  involves  three  team 
actions:  covering  for  others,  coaching,  and  antidpation. 

Covering  for  others  involves  team  members  stepping  appropriately  outside  of,  or  adding  to, 
their  assigned  team  roles  or  functions  in  order  to  help  the  team  reach  its  goals. 

Coaching  occurs  when  a  team  member  helps  another  team  member  achieve  a  greater  level  of 
individual  proficiency.  The  entire  team  benefits  from  coaching  because  the  team  moves  up 
to  a  more  advanced  level  of  performance  by  improving  the  capabilities  of  its  members. 

Anticipation  involves  anticipating  the  information  needs  of  members  who  have  become 
overloaded  by  uncertain  conditions  and  time  constraints.  It  also  involves  members  offering 
information  and  help  to  reduce  the  burden  of  overloaded  members. 

Planning.  Compensating  is  evident  in  two  ways  during  planning.  First,  to  develop  a  plan, 
members  of  the  team  carry  out  specific  roles  and  functions.  During  planning  tasks  you  can 
watch  members  cover  for  others,  coach  others,  and  anticipate  the  information  needs  of 
others.  Second,  the  plan  itself  may  highlight  areas  of  weakness  and  point  to  times  when 
compensation  will  be  necessary. 

Execution.  During  execution  members  may  temporarily  assist  other  members,  provide  on- 
line  coaching,  and  anticipate  and  provide  the  information  needs  of  other  team  members.  It  is 
important  to  distinguish  between  providing  needed  versus  unneeded  information  as  well  as 
supplying  appropriate  information  at  the  time  it  is  needed. 


Indicators  of  effective  performance 

•  When  the  BC  didn’t  know  the  strengths  of  the  company’s  members  he  watched  their 
performance. 

•  When  the  BC  learned  a  new  technique,  he  passed  it  on  to  the  team  members  who 
could  benefit  from  the  new  skill. 

•  The  BC’s  aide  provided  information  to  the  chief  as  reminders  that  an  action  needed  to 
be  taken. 

•  When  an  officer  didn’t  do  something  that  he  is  required  to  do,  the  battalion  BC 
pointed  out  the  reasons  it  is  a  requirement. 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Envisioning  goals 

This  dimension  refers  to  the  extent  to  which  members  of  the  team  understand  the  goal  The 
three  aspects  of  this  dimension  are:  identifying  the  goals,  ensuring  that  aU  members  share  a 
common  understanding  of  the  goals,  and  establishing  milestones  for  meeting  the  goal^ 
advanced  teams,  understanding  occurs  early  and  continues  throughout  the  operation  of  the 
task.  When  goals  shift  during  task  execution,  or  when  one  goal  takes  priority  over  another, 
the  new  goals  are  communicated  to  team  members. 

Planning.  When  developing  a  plan  for  task  execution,  the  team  leader  must  articulate  the 
overall  goals  making  sure  that  all  members  of  the  team  understand  them. 

Execution.  When  goals  are  modified,  completely  abandoned,  or  one  takes  priority  oyer 
another  then  these  changes  must  be  communicated  to  and  understood  by  the  appropriate 
team  members.  Appropriate  members  include  those  people  whose  tasks  may  be  affected  by 
the  different  goal.  In  tasks  that  are  characterized  by  time  stress  and  uncertainty,  the 
continual  assessment  of  the  situation  allows  the  leader  to  know  if  the  goals  are  being  attained. 

Mpfhnds  for  envisioning  goals.  The  key  to  this  dimension  is  the  leaders’  articulation  ^d  the 
teams’  understanding  of  the  goals.  Leaders  can  ensure  this  understanding  by  asking  the 
members  to  repeat  the  goals. 

<1 

Indicators  of  effective  performance 

•  The  immediate  goal  was  to  develop  a  size-up  of  the  situation.  Obvious  problems  are 
dealt  with  first. 

•  The  team  responded  with  the  following  order  of  goals  in  mind: 

a)  safety  of  firefighters  and  occupants  (assessed  the  integrity  of  the  structure  and  the 

extent  of  the  fire)  (searched  for  occupants) 

b)  fire  extension 

c)  prope^  (avoid  excessive  property  damage) 

d)  leave  the  community  with  a  good  attitude  about  the  fire  department 

•  The  first-on-the-scene  officer  responded  with  the  above  goals  in  mind. 

•  When  someone  was  unclear  about  the  leader’s  intent  the  person  sought  clarification. 

The  value  of  this  dimension  to  team  performance.  When  the  whole  team  understods  the 
goal,  all  members  can  operate  within  common  objectives  and  can  improvise  within  changing 
conditions  of  the  dynamic  situation. 
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CATEGORY:  TEAM  DECISION  MAKING 

Dimension:  Assessing  the  situation 


Situation  assessment  concerns  the  team’s  ability  to  construct  a  hypothesis  about  what  is 
actually  taking  place  in  their  real-world  environment.  Team  members  can  input  information 
that  will  allow  an  accurate  situation  assessment.  The  team  also  must  work  from  a  similar 
perspective  of  the  situation.  These  assessments  must  be  clearly  communicated  and 
understood  by  all  members.  Real  problems  can  arise  if  different  team  members  have 
^iffPTpnt  perspectives  about  what  is  occurring. 

Planning.  During  planning  there  are  two  critical  aspects  of  sharing  situation  assessment. 

The  first  is  Divergence.  Divergence  involves  encouraging  team  members  to  articulate 
different  perspectives  or  interpretations  about  what  is  happening  in  the  situation.  It  also 
involves  seeking  views  from  a  variety  of  team  members.  The  second  is  Convergencg. 
Convergence  involves  the  elimination  of  perspectives  that  are  ui^ely,  and  apeement  on  one 
assessment  of  the  situation.  Converging  is  a  process  of  harnessing  various  views  of  the 
situation  to  arrive  at  an  assessment  that  reflects  ground  truth.  Once  an  assessment  is  inade, 
it  is  important  for  team  members  to  work  from  a  similar  assessment.  This  can  be  attained 
through  the  use  of  maps,  diagrams,  messages,  and  meetings  to  ensure  that  team  members  are 
viewing  the  situation  similarly. 

Execution.  During  task  execution,  it  is  important  to  form  an  accurate  size-up  and  to  make 
sure  that  all  members  know  the  assessment.  Sharing  the  situation  assessment  will  make  it 
easier  for  members  to  recognize  when  something  is  occurring  that  indicates  ^  inaccurate 
•  assessment.  These  exceptions  are  then  shared  with  the  rest  of  the  team.  This  perrmts  the 
team  to  adjust  the  situation  assessment  on  line  to  more  closely  match  ground  truth. 


Indicators  of  effective  performance 


•  The  following  factors  were  included  in 

Structure: 

□  Locaition 

□  Number  of  stories 

□  Occupancy  status 

□  Sprinkler  system 

Safety: 

□  If  structure  is  engulfed  in  flames, 
the  BC  does  an  outside  size-up, 
looking  for  cracks  in  structure, 
parts  of  the  building  falling  off. 

•  Frequent  status  reports  were  given. 


the  size-up: 

□  Type  of  construction  material 

□  Type  of  habitat 

□  Size 

Contents  of  the  building: 

□  Hazardous  materials 

□  rubbish 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Focusing  attention  on  range  of  factors 

This  dimension  refers  to  the  team’s  ability  to  focus  its  attention  on  the  most  relevant  cues.  It 
specifically  involves  the  team’s  awareness  of  all  the  factors  that  must  be  considered  during 
problem  solving  and  decision  making.  Focusing  on  an  appropriate  range  or  number  of 
factors  is  key  to  advanced  team  performance.  The  team  should  not  consider  so  many  that  it 
becomes  paralyzed  or  so  few  that  the  solution  results  in  error. 

Planning.  During  the  planning  stage  the  team  identifies  the  factors,  conditions,  or  elements 
that  must  be  considered  at  the  time  of  the  incident  and  evaluated  throughout  the  incident. 
Making  sure  all  team  members  are  aware  of  and  understand  these  factors  is  an  important 
aspect  of  this  dimension. 

Execution.  The  factors  are  considered  at  the  time  of  the  incident  and  are  evaluated 
throughout  the  incident.  When  factors  different  from  the  ones  originally  considered  are 
relevant  to  a  specific  situation,  these  must  be  communicated  to  the  appropriate  team 
members.  The  factors  that  must  be  considered  during  execution  are  often  in  the  form  of 
check  lists,  but  not  always. 

Indicators  of  effective  performance 

•  The  BC  did  not  go  inside  the  building  unless  he  was  the  first  person  on  the  scene. 
(The  danger  is  that  he  may  acquire  the  urge  to  focus  on  minutiae.) 

•  The  time  of  day  and  the  day  of  the  week  informed  the  members  about  the  occupancy 
of  the  building.  (For  instance,  if  it  is  Saturday,  then  the  possibility  of  children  being 
home  is  high.  Also  on  Saturday,  people  are  usually  home  and  there  is  a  greater 
chance  of  people  being  out  on  the  street  watching  if  the  building  is  really  on  fire.  At 
l;30pm,  people  are  usually  not  sleeping,  so  someone  in  the  house  should  be  awake.) 

The  value  of  this  dimension  to  team  performance.  Firefighters  must  rely  on  a  great  deal  of 
knowledge  to  accurately  size  up  a  situation.  They  must  also  consider  many  factors  which 
change  from  situation  to  situation  depending  upon  the  nature  of  the  situation.  In  firefighting, 
a  plan  of  action  can  usually  fall  pretty  easily  from  an  accurate  size-up.  Knowledge  of  the 
factors  that  allow  an  accurate  size-up  is  important  to  team  performance. 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Focusing  attention  on  time  horizon 

This  dimension  refers  to  the  team’s  ability  or  skiU  to  look  into  the  future  in  order  to  be 
proactive  and  stay  ahead  of  the  fire.  Time  horizon  involves  the  team’s  awareness  of  ^o 
rimft  elements:  upcoming  events  and  the  amount  of  time  it  takes  to  accomplish  an  action. 

An  advanced  team: 

•  will  anticipate  upcoming  events  and  plan  activities  accordingly, 

•  has  knowledge  of  the  amount  of  time  it  takes  to  accomplish  an  action  and  plans  and 
assigns  tasks  accordingly,  and 

•  understands  lag  times  to  avoid  getting  behind  the  timeline. 


Time  horizon  is  different  from  time  management.  Time  horizon  deals  with  reaction  time 
while  time  management  deals  with  real-time  planning. 

Planning.  During  the  planning  stage,  the  team  will  identify  and  discuss  the  amount  of  time  it 
to  complete  an  activity,  e.g.,  "remember,  it  takes  10  minutes  to  move  X.  Making 
sure  all  team  members  are  aware  of  and  understand  these  time  issues  is  an  important  aspect 
of  this  dimension. 

Execution.  During  execution,  this  is  evident  when  the  team  has  failed  to  anticipate  and  work 
with  the  lag  times  that  often  occur  during  real-world  task  execution.  For  example,  a  team 
may  start  doing  something  when  it  needs  to  be  done  rather  than  taking  into  consideration  the 
fact  that  it  will  take  10  minutes  to  have  the  effect  they  want  right  now. 

Indicators  of  effective  performance 

•  The  building  information  was  on  hand. 

•  Everyone  reacted  quickly. 

•  Information  about  the  layout  of  the  streets  was  shared  with  the  drivers. 

•  Members  of  the  team  anticipated  the  amount  of  time  it  takes  for  equipment  to  arrive. 

•  The  BC  anticipated  the  amount  of  time  it  takes  for  equipment  to  arrive. 

The,  value  of  this  dimension  to  team  performance.  The  commander  must  plan  to  fight  the 
fire  that  will  be  there  in  10  minutes  rather  than  fight  the  fire  he  sees  when  just  arriving  on 
the  scene.  Understanding  time  horuxin  issues  will  increase  the  likelihood  that  the  team  will 
be  able  to  achieve  this. 
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CATEGORY:  TEAM  DECISION  MAKING 

Dimension:  Managing  uncertainty 

This  dimension  refers  to  the  team’s  ability  to  recognize  and  communicate  uncertainty  about 
the  information  that  impacts  situation  assessment.  If  there  is  uncertainty  about  the  nature  of 
a  situation,  then  the  advanced  team  actively  attempts  to  clarify  the  uncertainty,  develop  plans 
that  are  flexible  enough  to  allow  rapid  shifts  in  activity  as  the  situation  becomes  clearer,  or 
hold  onto  knowledge  of  the  uncertainty  until  it  can  be  resolved. 

Plannine.  During  the  planning  stage  the  team  may  identify  what  they  know  and  what  is  still 
uncertain,  and  make  sure  that  appropriate  members  are  aw^  of  the^  gaps  in  the  team’s 
knowledge  base.  Teams  may  also  assign  specific  information-gathering  tasks  to  attempt  to 
reduce  this  uncertainty.  During  planning,  the  key  to  effective  performance  is  to  identify  the 
gaps  in  their  knowledge,  to  make  sure  appropriate  members  know  where  these  gaps  are,  and 
to  anticipate  how  they  might  gather  this  information  on  line. 

Execution.  During  execution,  look  for  the  team  to  occasionally  communicate  not  only  what 
they  see  or  know  but  also  what  they  don’t  know.  It  should  also  be  possible  to  see  team 
members  remind  other  team  members  of  uncertainty  when  appropriate. 

Indicators  of  effective  performance 

•  The  BC  reviewed  the  Vital  Buildings  Information  sheet  as  he  was  going  to  an  incident 
in  a  building  that  he  is  unfamiliar  with. 

•  Information  was  shared  about  the  kind  of  buildings  that  surrounded  the  address. 

•  Information  was  shared  about  what  they  didn’t  know  about  the  building. 

•  Members  clarified  the  location  of  the  incident  when  they  were  unsure. 

•  Because  the  BC  was  not  up-to-date  on  latch-key  laws,  he  waited  for  a  police  officer  to 
arrive  before  he  broke  into  the  building  to  get  a  little  boy  out.  Meanwhile,  the  ladder 
officer  noticed  fire  hazards  in  the  building,  giving  them  reason  to  rescue  the  boy. 
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CATEGORY:  TEAM  SELF  MANAGEMENT 

Dimension:  Monitoring/adjusting 

This  dimension  refers  to  the  team’s  ability  to  examine  itself  for  signs  of  both  effective  and 
ineffective  teamwork  behaviors  and  change  or  reward  behaviors  at^rdingly.  The  pi^se  of 
monitoring  is  to  recognize  how  the  team  is  operating  and  make  adjustments.  Recognizing 
effective  behaviors  increases  the  team’s  ability  to  use  them  in  the  future.  Recognizing 
ineffective  processes  and  behaviors  and  changing  them  increases  the  team’s  performance  on 

line. 

Planning.  During  the  planning  stage,  observations  will  focus  on  how  the  team  is  monitoring 
and  adjusting  team  performance  within  the  context  of  the  planning  task.  During  this  stage, 
you  can  watch  the  team  to  see  if  it  is  verifying  that  they  have  identified  and  communicated 
their  resources,  roles  and  functions,  goals,  and  uncertainties.  Also,  the  team  wants  to 
monitor  members’  engagement,  sharing  of  situation  ^sessment,  compensation.  When  the 
team  recognizes  inefficient  team  processes,  it  can  adjust  them  to  make  them  more  efficient. 

Execution.  During  execution,  the  team  will  monitor  their  performance  within  the  context  of 
the  task  they  are  executing.  When  there  is  a  recognition  of  inefficiently  performed  team 
processes,  or  a  complete  lack  of  team  processes,  the  team  attempts  to  reinstate  these 
processes,  e.g.,  roles  &  functions,  goals,  assessing  situation  assessment,  compensation,  etc. 

Indicators  of  effective  performance 

•  When  the  BC  arrived  on  the  scene,  an  officer  told  him  what  they  had  been  doing  and 
what  they  intended  to  do.  The  BC  determined  if  they  should  continue. 

•  The  BC  monitors  procedures  and  gives  direction  to  ensure  they  are  being  followed. 
(For  instance,  a  ladder  company  was  assisting  the  police  on  a  search  for  guns  atop 
roof.  In  an  assist  police  call,  the  ladder  company  is  not  to  put  their  people  at  risk. 

In  this  case,  the  BC  noticed  that  the  firefighter  was  climbing  the  ladder  and  told  him 
that  his  job  was  only  to  usher  them,  not  to  do  their  job  for  them.) 
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CATEGORY:  TEAM  SELF  MANAGEMENT 

Dimension:  Time  management 

This  dimension  refers  to  the  team’s  ability  to  meet  goals  before  deadlines  overtake  them. 
Managing  time  includes  sequencing  sub-tasks  so  that  output  from  one  task  conn^ts  where 
and  when  it  should  as  input  to  the  next  task  and  allocating  the  right  amount  of  time  to  the 
most  important  tasks. 

Planning.  During  the  planning  stage,  time  management  occurs  in  two  ways.  First, 
managing  the  time  of  the  planning  process  itself,  keeping  the  team  on  schedule  so  it  does  not 
run  out  of  time  to  accomplish  various  planning  tasks  and  is  ready  on  time  for  the  execution 
stage.  Second,  working  on  anticipating  the  time  management  issues  for  the  execution  phase 
such  as  attempting  to  ascertain  that  they  are  being  realistic  about  what  they  need  to 
accomplish  in  the  time  available  during  execution. 

Execution.  Look  for  real-time  timekeeping.  On  schedule,  off  schedule,  been  at  this  task  X 
long,  etc. 

Indicators  of  effective  performance 

•  BC  Aide  watched  the  clock  to  inform  the  BC  how  long  people  had  been  looking  for 
the  fire.  (For  instance,  if  firefighters  have  been  looking  for  the  fire  for  15  minutes 
and  no  one  has  found  anything,  but  the  smoke  is  getting  thicker,  then  the  Aide  will 
inform  the  BC  of  this.) 

•  The  BC  Aide  watched  the  clock  to  time  how  long  people  had  been  inside  the  burning 
building  and  reported  this  to  the  BC. 
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A  Model-Based  Team  Decision-Making  and  Performance  Assessment 
Instrument:  Development  and  Evaluation 

Handoff  and  Instructional  Package 


Volume  II 


TEAM  PERFORMANCE  MEASUREMENT  PACKAGE 


This  package  contains  4  parts: 

PART  1 .  INTRODUCTION 

PART  2.  EXPLANATION  OF  TEAM  PERFORMANCE  MODEL 

PART  3.  ASSESSMENT  INSTRUMENT 

PART  4.  ACCOMPANYING  MATERIALS  (left  folder  pocket) 

•  descriptions  of  incidents 

•  assessment  forms 

Please  read  the  Introduction  and  Parts  2  and  3  before  filling  out  the 
assessment  forms. 

Thank  you  in  advance. 
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PARTI.  INTRODUCTION 


If  a  group  of  highly  competent  individuals  is  not  enough  to 
guarantee  efficient  teamwork,  what  is?  We  have  all  been  in  teams 
that  have  performed  with  varying  degrees  of  efficiency.  Sometimes 
we  blame  the  problems  on  the  nature  of  the  task  and  sometimes  we 
blame  certain  individuals  on  the  team.  Our  belief,  however,  is  that 
certain  processes  exist  that  facilitate  the  coordination  of  individual 
tasks  and  the  achievement  of  advanced  team  performance.  We  also 
believe  that  these  processes  can  be  enhanced  once  the  team 
recognizes  them.  This  package  has  been  designed  to  help  people 
understand  and  recognize  these  team  processes. 

We  have  developed  this  package  for  anyone  who  is  responsible  for 
accomplishing  tasks  through  teamwork.  This  particular  package 
was  designed  for  Battalion  Chiefs  in  Fire  Departments  to  help  them 
view  team  performance  in  ways  that  were  not  apparent  before.  Its 
purpose  is  to  describe  a  way  of  looking  at  teams  to  see  where  they 
are  succeeding  and  where  they  could  improve.  This  new  team 
perspective  will  allow  you  to  develop  interventions  that  will  improve 
team  performance.  These  interventions  can  be  made  either  during 
an  incident  to  enhance  team  performance  or  during  after-incident 
reviews  to  emphasize  lessons  learned. 

The  model  of  team  performance  we  have  developed  is  based  on 
findings  from  five  years  of  studying  various  teams:  helicopter  and 
cockpit  crews,  firefighting  teams  suppressing  forest  fires.  Army 
command  and  control  teams  from  Battalions,  Corps,  Divisions,  and 
echelons  above  the  Corps  level  at  the  Army  War  College  and  the 
National  Defense  University. 

During  our  interactions  with  team  members  we  have  asked  what 
they  think  makes  for  good  team  performance.  We  consistently  get 
two  answers:  good  communication  and  good  coordination.  One 
study  with  high  ranking  fire  safety  officers  during  an  emergency 
simulation  showed  that  more  efficient  teams  were  better  in  task 
distribution  and  coordination.  Our  goal  is  to  identify  the  behaviors 
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and  processes  that  ensure  effective  communication,  coordination, 
and  task  distribution. 

We  have  also  heard  that  good  team  performance  is  directly  related 
to  how  well  team  members  get  along.  But  that  is  not  the  focus  of 
this  effort.  We  believe  that  processes  like  knowledge  of  the  roles 
and  functions  that  must  be  fulfilled,  understanding  of  and 
commitment  to  the  common  goal,  and  an  adequate  assessment  of 
the  situation  are  more  important  to  effective  team  performance  than 
the  personalities  of  its  members.  While  this  package  is  not  designed 
to  help  team  members  "get  along  better,"  It  can  help  the  team 
understand  the  impact  of  personality  conflicts  on  the  processes  of 
team  performance. 

Using  the  assessment  instrument  contained  in  this  package  involves 
looking  at  the  performance  of  your  team  along  14  separate 
dimensions.  To  help  you  learn  to  apply  it,  we  have  provided 
examples  of  firefighting  incidents  in  which  you  were  involved.  We 
ask  you  to  examine  and  rate  your  team's  performance  during  these 
incidents  on  the  14  dimensions.  Before  you  do  that,  we  ask  you  to 
read  the  overview  of  the  four  categories  of  team  performance  in 
hopes  of  helping  you  learn  to  view  team  performance  in  a  more 
structured  way.  The  four  categories  make  up  the  model  of  team 
performance  on  which  the  assessment  is  based.  They  are:  Team 
Resources,  Team  Identity,  Team  Decision  Making,  and  Team  Self 
Management.  These  will  be  explained  in  greater  detail  in  Part  2  of 
this  package.  The  ideas  contained  in  this  package  should  not  be 
foreign  to  you  since  it  was  validated  by  1 1  Battalion  Chiefs  in  the 
Philadelphia  Fire  Department. 

Part  2  of  this  package  provides  the  overview.  Part  3  contains 
instructions  for  using  the  assessment  tool.  After  you  have  read 
through  Parts  2  and  3,  fill  out  the  assessment  forms  contained  In 
Part  4  in  the  front  pocket  of  this  folder. 
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PART  2.  EXPLANATION  OF  TEAM  PERFORMANCE 
MODEL 

The  purpose  of  this  team  performance  framework  is  to  redirect  your 
attention  from  focusing  solely  on  individual  performance  to  seeing 
the  coordination  efforts  of  team  members.  There  are  four  major 
categories  of  advanced  team  performance.  They  are: 

Team  Resources 
Team  Identity 
Team  Decision  Making 
Team  Self  Management 

In  this  part  we  describe  the  four  major  categories.  Each  category  is 
further  divided  into  several  dimensions  that  provide  a  clearer 
representation  of  the  category.  You'll  find  these  dimensions  listed 
within  the  category  they  correspond  to. 

TEAM  RESOURCES 

This  category  includes  the  resources  that  are  available  to  the  team 
for  getting  the  job  done.  Resources  include  people,  equipment,  and 
standard  procedures.  A  team  that  knows  the  resources  available  to 
it  and  the  capabilities  of  these  resources  for  getting  the  job  done  will 
be  in  a  better  position  to  use  the  resources  appropriately.  The  team 
must  make  sure  that  the  resources  available  to  It  and  the  capabilities 
of  each  are  communicated  to  the  appropriate  team  members. 

The  four  dimensions  of  Team  Resources  are: 

Member  resources 
Leadership  resources 
Non-human  resources 
Basic  procedures 

We  believe  that  if  team  members  are  aware  of  these  resources  they 
have  a  greater  chance  of  taking  maximum  advantage  of  them.  The 
next  category  describes  how  this  can  happen. 
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TEAM  IDENTITY 


Teams  develop  a  sense  of  identity.  There  is  a  clear  shift  from 
individuals  who  are  focused  on  doing  their  individual  jobs  to  an 
integrated  team  whose  members  are  in  tune  with  what  it  takes  to 
achieve  the  collective  goals.  It  is  through  the  development  of  team 
identity  that  good  teamwork  begins  to  emerge. 

What  will  strengthen  team  identity?  Members'  knowledge  of  the 
roles  which  others  are  fulfilling  and  an  understanding  of  how  their 
own  role  fits  in  with  them.  There  are  a  number  of  ways  you  can  tell 
the  strength  of  a  team's  identity  or  commitment  to  the  common 
goal.  You  can  tell  by  watching  to  see  if  each  member  is  actively 
engaged  in  achieving  the  team  goal.  You  can  also  tell  by  the 
behaviors  of  members  who  are  encouraging  others  to  engage. 
Another  way  to  judge  a  team's  level  of  identity  is  by  looking  at  the 
compensation  behaviors  that  are  exhibited.  Compensation  behaviors 
include  members  stepping  appropriately  out  of  their  own  roles  and 
filling  another  role  if  the  need  exists.  You  can  also  judge  the  team's 
strength  of  identity  by  watching  to  see  if  members  who  have 
greater  skills,  knowledge,  and  experience  are  coaching  members 
who  are  at  a  lower  level.  Another  indication  of  strength  is  whether 
or  not  team  members  can  anticipate  the  needs  of  other  members 
and  fill  those  needs  during  stressful  situations  without  being  asked. 

The  three  dimensions  of  team  identity  are: 

Defining  roles  and  functions 

Engaging 

Compensating 

Firefighters  face  situations  with  different  task  responsibilities  and 
must  rely  on  the  knowledge  and  expertise  of  other  more  experienced 
firefighters.  Team  identity  is  about  taking  maximum  advantage  of 
the  members'  knowledge  and  experiences  and  increasing  both 
whenever  possible. 
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TEAM  DECISION  MAKING 


Firefighting  situations  are  made  up  of  many  factors  that  are 
continually  changing  (i.e.,  the  nature  of  fire  and  the  firefighters' 
impact  on  the  situation  as  they  begin  to  fight  the  fire).  In  situations 
that  are  characterized  by  risk  and  time  pressure  it  is  important  for 
teams  to  learn  how  to  perceive  what  it  sees  on  the  fireground  and 
to  respond  effectively.  This  category  points  to  the  processes  which 
allow  a  team  to  be  highly  responsive  to  changing  conditions.  Our 
definition  of  decision  making  is  broad.  It  includes  assessing  the 
situation,  deciding  upon  a  course  of  action,  and  monitoring  the 
situation  to  make  sure  the  course  of  action  is  meeting  the  goals. 

The  five  dimensions  that  represent  this  category  are: 

Envisioning  goals 
Assessing  the  situation 
Focusing  attention  on  range  of  factors 
Focusing  attention  on  time  horizon 
Managing  uncertainty 

Teams  share  common  goals,  but  do  they  also  share  a  common 
understanding  of  the  goal?  Firefighting  teams  learn  the  goals  over 
time  as  officers  watch  their  company's  performance  toward  those 
goals  and  then  provide  constructive  feedback.  This  is  one  good 
strategy  for  helping  the  teams  envision  goals.  We  will  discuss 
others  in  the  assessment  tool. 

In  order  to  be  responsive  in  a  dynamic  environment  teams  must 
assess  the  situation  to  form  a  quick  and  accurate  size-up.  This 
search  depends  on  the  team's  ability  to  scan  a  sufficiently  broad 
range  of  factors  to  detect  important  patterns,  and  to  focus  its 
attention  far  enough  into  the  future  to  be  able  to  react  to  events. 

Recognizing  and  managing  what  isn't  known  about  a  situation  is 
also  important  for  improving  team  responsiveness.  For  instance.  If 
the  integrity  of  the  building  has  not  yet  been  assessed,  but  civilians 
are  trapped  Inside,  then  it  may  be  necessary  to  gather  more 


6 


information  about  the  building  before  sending  someone  in. 
Uncertainty  is  managed  by  the  assignment  of  information-gathering 
tasks  and  holding  back  until  the  uncertainty  is  clarified. 

TEAM  SELF  MANAGEMENT 

The  final  category  of  team  performance  concerns  the  team's  ability 
to  oversee  its  performance  as  a  team.  Team  self  management  is  a 
monitoring  and  adjustment  function.  A  team  needs  to  monitor  what 
is  occurring  and  note  differences  from  what  is  expected,  so  that  it 
can  adjust  appropriately.  In  fact,  if  the  team  monitors  its 
performance  on  the  three  categories  just  described.  Team 
Resources,  Team  Identity,  and  Team  Decision  Making,  and  makes 
adjustments  when  necessary,  the  team  will  advance  in  its  level  of 
teamwork.  Adjustments  to  team  performance  can  occur  during  task 
execution  and/or  after  task  execution  in  meetings  to  discuss  the 
lessons  learned. 

The  dimensions  that  represent  team  self  management  are: 

Monitoring/adjusting 
Time  management 

Each  of  the  four  categories  just  described  consist  of  dimensions  that 
indicate  advanced  performance.  These  dimensions  are  defined  in 
Part  3  of  this  package. 
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PART  3.  ASSESSMENT  INSTRUMENT 

INSTRUCTIONS  FOR  USING  THE  ASSESSMENT 
INSTRUMENT 

The  instrument  consists  of  three  parts: 

1 .  Definition  of  the  team  performance  dimensions. 

2.  Examples  of  firefighting  incidents  you  were  involved  in. 

3.  Assessment  forms  for  rating  team  performance. 

There  are  three  steps  to  applying  this  instrument.  First,  become 
familiar  with  the  detailed  descriptions  of  the  14  dimensions  of  the 
team  performance  model.  Second,  refamiliarize  yourself  with  the 
firefighting  incidents  (left  folder  pocket).  Note  that  these  are 
incidents  you  were  involved  in  when  we  rode  with  you  in  May, 

1994.  Attached  to  each  incident  is  a  set  of  assessment  forms.  The 
third  stop  is  to  assess  each  incident  along  the  14  dimensions.  The 
next  paragraph  provides  more  detailed  instructions. 

In  the  left  folder  pocket  you  will  find  packets  of  papers  stapled 
together.  The  first  page  is  a  description  of  the  incident  in  which  you 
were  involved.  The  remaining  pages,  which  are  attached  to  each 
*  incident,  make  up  a  set  of  assessment  forms.  You  will  use  these 
assessment  forms  to  rate  your  team's  performance. 

Before  you  rate  the  incident  for  performance  on  a  particular 
dimension,  please  read  the  definition  of  the  dimension  and  make 
sure  you  understand  it.  The  definitions  of  each  of  the  1 4 
dimensions  are  listed  next.  With  each  definition,  we  have  included 
examples  of  behaviors  to  help  clarify  the  meaning  of  each 
dimension.  Our  intent  for  listing  the  behaviors  is  merely  to  clarify. 
They  are  not  intended  to  be  comprehensive.  That  is,  you  may  think 
of  examples  of  behaviors  not  included  in  the  list. 
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Our  definitions  include  what  team  performance  might  look  like 
during  a  planning  session  and  what  it  might  look  like  during  task 
execution.  It  takes  coordinated  team  effort  to  develop  a  plan  for 
approaching  a  task  or  a  project  just  as  it  does  to  execute  a  task. 

For  instance,  when  you  fill  out  a  Vital  Building  Information  form  on  a 
high-rise  building,  you  will  consult  with  others  to  make  sure  you 
include  all  pertinent  data  and  create  appropriate  approaches  to  the 
fire.  The  team  performance  dimensions  are  as  important  during 
planning  as  they  are  during  task  execution,  but  it  takes  a  slightly 
different  orientation  to  recognize  the  team  performance  behaviors. 
Within  the  description  of  the  behavioral  dimensions,  we  explain  the 
behaviors  for  planning  and  for  execution. 

As  you  read  about  and  understand  the  meaning  of  a  dimension, 
please  rate  the  incident  using  the  corresponding  assessment  form 
for  that  dimension. 

Reminder:  Call  Molly  or  Marv  (513-873-8166)  if  you  need  any 
further  clarification. 

DEFINITIONS  OF  DIMENSIONS 

See  the  following  14  pages. 
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CATEGORY:  TEAM  RESOURCES 
Dimension:  Member  resources 

This  dimension  refers  to  the  team's  understanding  of  its  members'  capabilities. 
Capabilities  include  the  skills,  knowledge,  mental  and  physical  abilities, 
experiences,  and  personalities.  What  is  critical  to  team  performance  is  not  so 
much  knowing  how  "good"  the  individuals  are,  but  knowing  what  resources  the 
team  has  to  work  with.  By  having  a  good  understanding  of  the  member  resources, 
the  team  will  be  better  able  to  match  these  resources  to  the  demands  of  the  task. 

Planning.  When  planning  who  will  do  ajparticular  task,  the  team  should  assess 
members'  strengths  and  weaknesses  and  make  sure  all  members  understand  what 
is  available.  All  members  should  know  who  brings  what  strengths. 

Execution.  Before  executing  a  task,  the  team  should  try  to  gain  an  understanding 
of  its  member  resources.  You'll  see  this  dimension  during  task  execution  if 
members  are  monitoring  the  on-going  status  of  team  members  and  making 
appropriate  adjustments  in  the  event  of  fatigue,  injuries,  or  inability  to  do  the  job. 

Indicators  of  effective  performance  in  the  firefighting  domain. 

•  BC  ascertains  who  is  in  each  company  during  a  given  shift  prior  to  an 
incident. 

•  BC  ascertains  the  competence  level  of  individuals  in  the  battalion. 

•  BC  listens  to  the  radio  and  knows  which  companies  are  responding  to  the 
incident. 

•  BC  ascertains  who  can  and  who  cannot  handle  an  incident.  This  includes 
the  ladder  and  engine  companies,  the  medics,  and  the  officers. 

The  value  of  this  dimension  to  team  performance.  One  Battalion  Chief  was  called 
to  a  fire  across  town.  On  his  way  there,  the  engine  company  officer  placed  it 
under  control.  The  Battalion  Chief  believed  the  officer  put  it  under  control  because 
he  knew  it  was  a  long  way  for  the  Battalion  Chief  to  go;  he  was  not  yet 
completely  confident  in  this  officer's  skills.  The  Battalion  Chief  ended  up  going 
anyway  and  spent  an  hour  on  the  fireground.  The  Battalion  Chief  s  knowledge  of 
the  way  this  officer  worked  increased  the  operating  efficiency  of  the  team. 

Methods  for  learning  about  team  members.  The  capabilities  of  members  can  be 
assessed  through  observations,  informal  discussions,  interviews,  education 
records,  and  resumes.  You  may  also  employ  more  formal  assessment  methods 
such  as  psychometric,  intelligence,  or  skills  tests. 


10 


CATEGORY:  TEAM  RESOURCES 
Dimension:  Leadership  resources 

This  dimension  is  about  team  members'  knowledge  of  their  leaders'  capabilities 
and  preferences. 

Planning.  The  team  should  discuss  the  strengths  and  limitations  in  its 
leadership  resources  and  be  clear  about  its  leadership  needs. 

Execution.  Before  executing  a  task  the  team  should  have  knowledge  of  who 
the  leaders  are  and  an  understanding  of  how  they  will  operate.  If  modifications 
to  leadership  are  made  during  execution,  the  changes  need  to  be  communicated 
to  the  whole  team. 

Indicators  of  effective  performance. 

•  BC  understands  what  the  officers  in  each  company  can  and  cannot 
handle. 

•  BC  determines  who  is  in  each  company  during  a  given  shift. 

•  If  the  officer  of  the  ladder  or  engine  company  arrives  on  the  scene  before 
the  BC,  all  are  clear  about  who  has  leadership  responsibility  until  the  BC 
arrives. 

•  When  the  BC  arrives,  he  receives  viable  information  and  then  takes 
command. 

•  When  the  Deputy  Chief  arrives,  he  slowly  eases  himself  into  the 
command  position.  Taking  over  immediately  may  disrupt  execution  of 
the  firefight. 

The  value  of  this  dimension  to  team  performance.  This  dimension  is  not  about 
the  strength  or  the  style  of  the  leader,  instead,  it  is  about  the  team's 
knowledge  of  the  leader's  strength  and  style.  Knowing  this  allows  the  team  to 
work  within  a  particular  style.  Leadership  functions  must  be  performed.  If  one 
leader  is  incapable,  unwilling,  or  too  busy  to  fulfill  the  functions,  then  these 
functions  must  be  distributed. 

Methods  for  learning  about  leadership.  Leadership  resources  can  be  assessed 
through  observation  and  discussion,  or  through  formal  methods  such  as 
leadership  profiles  and  interviews.  Firefighters  should  be  encouraged  to  get  to 
know  their  leaders. 
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CATEGORY:  TEAM  RESOURCES 
Dimension:  Non-human  resources 

This  dimension  refers  to  the  equipment,  apparatus,  tools,  supplies,  vehicles, 
and  computers/software  that  are  available  to  the  team  to  get  the  job  done.  The 
key  to  this  dimension  is  the  knowledge  members  have  about  what's  available, 
its  capabilities,  and  how  it  operates. 

Planning.  When  making  plans  to  execute  a  task,  the  team  should  verify  what 
resources  are  available  to  them,  and  identify  weaknesses  and  strengths.  This 
information  is  communicated  to  the  whole  team.  If  resources  are  added,  then 
the  team  needs  to  know  about  them. 

Execution.  The  team's  understanding  of  the  resources  available  to  them  should 
be  ascertained  prior  to  task  execution.  You  will  see  this  dimension  when 
changes  to  resources  occur  during  execution.  These  changes  need  to  be 
communicated  to  the  whole  team. 

Indicators  of  effective  performance. 

•  BC  checks  the  dispatch  notification  sheet  to  find  out  what  equipment  he 
will  have  available  to  work  with  on  the  scene.  This  is  referred  to  as  the 
company  rundown. 

•  Each  team  member  knows  what  equipment  will  be  arriving  at  a  given 
incident. 

•  Each  team  member  knows  which  resources  are  carried  in  the  various 
vehicles. 

The  value  of  this  dimension  to  team  performance.  Knowing  the  resources  that 
are  available  to  the  team  allows  members  to  take  steps  to  maximize  their  use. 
Conversely,  knowing  what  it  doesn't  have  to  work  with  allows  the  team  to 
work  around  the  absence.  For  example,  if  Rescue  One  is  not  available  to 
respond  to  a  job,  then  the  plan  of  action  must  include  resources  to  cover  the 
work  usually  performed  by  Rescue  One. 
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CATEGORY:  TEAM  RESOURCES 


Dimension:  Basic  Procedures 

This  dimension  refers  to  the  standard  procedures  that  are  in  place  to  provide  basic 
direction  and  processes  for  the  team  members.  Procedures  are  developed  to  guide 
the  team  to  more  efficient  and  effective  operation.  Procedures  are  enacted  by 
team  members  through  a  series  of  coordinated  activities.  Team  members  must  be 
aware  of  these  procedures  and  how  they  are  implemented.  The  smoothness  with 
which  procedures  are  implemented  indicates  the  team's  familiarity  with  its  basic 
procedures.  Familiarity  is  gained  through  both  study  and  practice. 

Planning.  The  team  may  rely  on  procedures  during  planning  sessions  to  direct  their 
activities.  Developing  procedures  that  will  be  used  during  task  execution  is 
different  from  implementing  the  procedures.  Basic  procedures  should  be 
established  in  advance  of  task  execution  and  task  execution  will  uncover  needed 
changes  or  additions  to  standard  procedures.  During  the  planning  stage  we  should 
see  the  team  verify  these  procedures,  communicate  them  to  the  team  members, 
and  incorporate  procedures  into  the  planning  process. 

Execution.  Characteristics  of  the  situation  will  inform  the  team  about  which 
procedure  should  be  implemented.  The  characteristics  and  the  subsequent 
procedure  are  established  prior  to  task  execution.  During  task  execution  a  well- 
informed  observer  can  watch  to  see  if  team  members  are  implementing  procedures 
appropriately.  If  a  procedure  is  no  longer  appropriate  for  the  characteristics  of  the 
task  and  it  is  altered  or  discarded,  then  these  changes  must  be  communicated  to 
the  whole  team. 

Indicators  of  effective  performance. 

•  When  the  team  responded  to  an  Assist  Police  call,  all  sirens  were  cut  4 
blocks  away  from  the  scene. 

•  When  a  Tactical  Box  call  went  out,  two  engines,  two  ladders,  and  one  BC 
responded. 

•  When  a  Full  Box  alarm  went  out,  four  engines,  two  ladders,  and  two  BCs 
responded. 

•  The  first  arriving  engine  took  the  front,  the  second  engine  took  the  rear,  the 
third  engine  took  the  front,  and  the  fourth  engine  took  the  rear.  The  first 
arriving  ladder  took  the  front,  the  second  ladder  took  the  rear.  The  first 
arriving  BC  took  the  front,  the  second  arriving  BC  took  the  rear. 

•  When  nothing  was  showing,  the  first  arriving  BC  called  for  reduced  speed  of 
incoming  engines. 

•  The  first  arriving  officer  assumed  leadership  responsibility  until  a  higher 
ranking  officer  arrived. 
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CATEGORY:  TEAM  IDENTITY 
Dimension:  Defining  roles  and  functions 

This  behavioral  dimension  refers  to  the  articulation  and  understanding  of  the 
task  responsibilities,  functions,  and  roles  of  every  other  member.  Once  roles 
and  functions  are  understood,  the  team  can  take  advantage  of  the  individual 
power  available  to  it.  Use  of  this  behavioral  dimension  will  facilitate 
coordination  efforts. 

Planning.  You  can  observe  this  dimension  during  planning  tasks  by  watching  to 
see  if  team  members  are  clarifying  their  roles  and  functions  and  making  sure 
that  everyone  understands  them.  Roles  and  functions  are  made  explicit. 

Execution.  Much  of  the  team's  understanding  of  their  roles  and  functions 
should  be  achieved  prior  to  task  execution.  If  the  team  is  required  to  make 
changes  in  roles  and  functions  on  line,  then  these  changes  must  be 
communicated  to  the  whole  team. 

Indicators  of  effective  performance. 

•  When  alarms  greater  than  a  tactical  box  are  called,  the  BCs  discuss  the 
roles  they  will  take. 

•  BCs  reminded  firefighters  about  their  role  in  an  assist  police  call.  That  is, 
firefighters  provide  the  means  for  the  police  to  do  their  job  and  do  not  get 
involved  in  the  incident.  This  prevents  firefighters  from  getting  near 
evidence  and  from  getting  into  situations  they  are  not  qualified  to  handle. 

•  Medics  let  firefighters  know  what  functions  they  can  serve  in  a  car 
accident  situation. 

•  It  is  well  known  that  the  BC  Aide  will  serve  as  a  time  keeper  to  keep 
track  of  how  long  people  have  been  inside  a  structure. 
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CATEGORY:  TEAM  IDENTITY 


Dimension:  Engaging 

This  behavioral  dimension  represents  the  extent  to  which  the  team  members 
participate  and  encourage  others  to  participate  in  the  team's  work  and  take 
responsibility  for  reaching  the  team  goals. 

Planning  and  Execution  behaviors  look  similar. 

Indicators  of  effective  performance. 

•  When  an  officer  is  handling  a  situation,  the  BC  does  not  take  over  the 
situation  unless  it  is  necessary.  If  he  does  take  over,  it  may  discourage 
the  officer  from  taking  responsibility. 
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CATEGORY:  TEAM  IDENTITY 
Dimension:  Compensating 

This  dimension  refers  to  the  movement  of  team  resources  to  cover  areas  where  the 
team  cannot  satisfy  its  roles,  functions,  and  responsibilities.  Compensation 
involves  three  team  actions:  covering  for  others,  coaching,  and  anticipation. 

Covering  for  others  involves  team  members  stepping  appropriately  outside  of,  or 
adding  to,  their  assigned  team  roles  or  functions  in  order  to  help  the  team  reach  its 
goals. 

Coaching  occurs  when  a  team  member  helps  another  team  member  achieve  a 
greater  level  of  individual  proficiency.  The  entire  team  benefits  from  coaching 
because  the  team  moves  up  to  a  more  advanced  level  of  performance  by  improving 
the  capabilities  of  its  members. 

Anticipation  involves  anticipating  the  information  needs  of  members  who  have 
become  overloaded  by  uncertain  conditions  and  time  constraints.  It  also  involves 
members  offering  information  and  help  to  reduce  the  burden  of  overloaded 
members. 

Planning.  Compensating  is  evident  in  two  ways  during  planning.  First,  to  develop 
a  plan,  members  of  the  team  carry  out  specific  roles  and  functions.  During 
planning  tasks  you  can  watch  members  cover  for  others,  coach  others,  and 
anticipate  the  information  needs  of  others.  Second,  the  plan  itself  may  highlight 
areas  of  weakness  and  point  to  times  when  compensation  will  be  necessary. 

Execution.  During  execution  members  may  temporarily  assist  other  members, 
provide  on-line  coaching,  and  anticipate  and  provide  the  information  needs  of  other 
team  members.  It  is  important  to  distinguish  between  providing  needed  versus 
unneeded  information  as  well  as  supplying  appropriate  information  at  the  time  it  is 
needed. 

Indicators  of  effective  performance. 

•  When  the  BC  didn't  know  the  strengths  of  the  company's  members  he 
watched  their  performance. 

•  When  the  BC  learned  a  new  technique,  he  passed  it  on  to  the  team  members 
who  could  benefit  from  the  new  skill. 

•  The  BC's  aide  provided  information  to  the  chief  as  reminders  that  an  action 
needed  to  be  taken. 

•  When  an  officer  didn't  do  somethmg  that  he  is  required  to  do,  the  BC 
pointed  out  the  reasons  it  is  a  requirement. 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Envisioning  goals 

This  dimension  refers  to  the  extent  to  which  members  of  the  team  understand  the 
goal.  The  three  aspects  of  this  dimension  are:  identifying  the  goals,  ensuring  that 
all  members  share  a  common  understanding  of  the  goals,  and  establishing 
milestones  for  meeting  the  goals.  In  advanced  teams,  understanding  occurs  early 
and  continues  throughout  the  operation  of  the  task.  When  goals  shift  during  task 
execution,  or  when  one  goal  takes  priority  over  another,  the  new  goals  are 
communicated  to  team  members. 

Planning.  When  developing  a  plan  for  task  execution,  the  team  leader  must 
articulate  the  overall  goals  making  sure  that  all  members  of  the  team  understand 
them. 

Execution.  When  goals  are  modified,  completely  abandoned,  or  one  takes  priority 
over  another,  then  these  changes  must  be  communicated  to  and  understood  by  the 
appropriate  team  members.  Appropriate  members  include  those  people  whose 
tasks  may  be  affected  by  the  different  goal.  In  tasks  that  are  characterized  by 
time  stress  and  uncertainty,  the  continual  assessment  of  the  situation  allows  the 
leader  to  know  if  the  goals  are  being  attained. 

Methods  for  envisioning  goals.  The  key  to  this  dimension  is  the  leaders' 
articulation  and  the  teams'  understanding  of  the  goals.  Leaders  can  ensure  this 
understanding  by  asking  the  members  to  repeat  the  goals. 

Indicators  of  effective  performance. 

•  The  immediate  goal  was  to  develop  a  size-up  of  the  situation.  Obvious 
problems  are  dealt  with  first. 

•  The  team  responded  with  the  following  order  of  goals  in  mind: 

a)  safety  of  firefighters  and  occupants  (assessed  the  integrity  of  the 
structure  and  the  extent  of  the  fire)  (searched  for  occupants) 

b)  fire  extension 

c)  property  (avoid  excessive  property  damage) 

d)  leave  the  community  with  a  good  attitude  about  the  fire  department 

•  The  first-on-the-scene  officer  responded  with  the  above  goals  in  mind. 

•  When  someone  was  unclear  about  the  leader's  intent  the  person  sought 
clarification. 

The  value  of  this  dimension  to  team  performance.  When  the  whole  team 
understands  the  goal,  all  members  can  operate  within  common  objectives  and  can 
improvise  within  changing  conditions  of  the  dynamic  situation. 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Assessing  the  situation 


Situation  assessment  concerns  the  team's  ability  to  construct  a  hypothesis  about 
what  is  actually  taking  place  in  their  real-world  environment.  Team  members  can 
input  information  that  will  allow  an  accurate  situation  assessment.  The  team  also 
must  work  from  a  similar  perspective  of  the  situation.  These  assessments  must  be 
clearly  communicated  and  understood  by  all  members.  Real  problems  can  arise  if 
different  team  members  have  different  perspectives  about  what  is  occurring. 

Planning.  During  planning  there  are  two  critical  aspects  of  sharing  situation 
assessment.  The  first  is  Divergence.  Divergence  involves  encouraging  team 
members  to  articulate  different  perspectives  or  interpretations  about  what  is 
happening  in  the  situation.  It  also  involves  seeking  views  from  a  variety  of  team 
members.  The  second  is  Convergence.  Convergence  involves  the  elimination  of 
perspectives  that  are  unlikely,  and  agreement  on  one  assessment  of  the  situation. 
Converging  is  a  process  of  harnessing  various  views  of  the  situation  to  arrive  at  an 
assessment  that  reflects  ground  truth.  Once  an  assessment  is  made,  it  is 
important  for  team  members  to  work  from  a  similar  assessment.  This  can  be 
attained  through  the  use  of  maps,  diagrams,  messages,  and  meetings  to  ensure 
that  team  members  are  viewing  the  situation  similarly. 

Execution.  During  task  execution,  it  is  important  to  form  an  accurate  size-up  and 
to  make  sure  that  ail  members  know  the  assessment.  Sharing  the  situation 
assessment  will  make  it  easier  for  members  to  recognize  when  something  is 
occurring  that  indicates  an  inaccurate  assessment.  These  exceptions  are  then 
shared  with  the  rest  of  the  team.  This  permits  the  team  to  adjust  the  situation 
assessment  on  line  to  more  closely  match  ground  truth. 


Indicators  of  effective  performance. 

•  The  following  factors  were  included  in 

Structure: 

□  Location 

□  Number  of  stories 

□  Occupancy  status 

□  Sprinkler  system 

Safety: 

□  If  structure  is  engulfed  in  flames, 
the  BC  does  an  outside  size-up, 
looking  for  cracks  in  structure, 
parts  of  the  building  falling  off.' 

•  Frequent  status  reports  were  given. 


the  size-up: 

□  Type  of  construction  material 

□  Type  of  habitat 

□  Size 

Contents  of  the  building: 

□  Hazardous  materials 

□  rubbish 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Focusing  attention  on  range  of  factors 

This  dimension  refers  to  the  team's  ability  to  focus  its  attention  on  the  most 
relevant  cues.  It  specifically  involves  the  team's  awareness  of  all  the  factors 
that  must  be  considered  during  problem  solving  and  decision  making.  Focusing 
on  an  appropriate  range  or  number  of  factors  is  key  to  advanced  team 
performance.  The  team  should  not  consider  so  many  that  it  becomes  paralyzed 
or  so  few  that  the  solution  results  in  error. 

Planning.  During  the  planning  stage  the  team  identifies  the  factors,  conditions, 
or  elements  that  must  be  considered  at  the  time  of  the  incident  and  evaluated 
throughout  the  incident.  Making  sure  all  team  members  are  aware  of  and 
understand  these  factors  is  an  important  aspect  of  this  dimension. 

Execution.  The  factors  are  considered  at  the  time  of  the  incident  and  are 
evaluated  throughout  the  incident.  When  factors  different  from  the  ones 
originally  considered  are  relevant  to  a  specific  situation,  these  must  be 
communicated  to  the  appropriate  team  members.  The  factors  that  must  be 
considered  during  execution  are  often  in  the  form  of  check  lists,  but  not  always. 

Indicators  of  effective  performance. 

•  The  BC  did  not  go  inside  the  building  unless  he  was  the  first  person  on 
the  scene.  (The  danger  is  that  he  may  acquire  the  urge  to  focus  on 
minutiae.) 

•  The  time  of  day  and  the  day  of  the  week  informed  the  members  about 
the  occupancy  of  the  building.  (For  instance,  if  it  is  Saturday,  then  the 
possibility  of  children  being  home  is  high.  Also  on  Saturday,  people  are 
usually  home  and  there  is  a  greater  chance  of  people  being  out  on  the 
street  watching  if  the  building  is  realJy  on  fire.  At  1 :30pm,  people  are 
usually  not  sleeping,  so  someone  in  the  house  should  be  awake.) 

The  value  of  this  dimension  to  team  performance.  Firefighters  must  rely  on  a 
great  deal  of  knowledge  to  accurately  size  up  a  situation.  They  must  also 
consider  many  factors  which  change  from  situation  to  situation  depending  upon 
the  nature  of  the  situation.  In  firefighting,  a  plan  of  action  can  usually  fall 
pretty  easily  from  an  accurate  size-up.  Knowledge  of  the  factors  that  allow  an 
accurate  size-up  is  important  to  team  performance. 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Focusing  attention  on  time  horizon 

This  dimension  refers  to  the  team's  ability  or  skill  to  look  into  the  future  in  order 
to  be  proactive  and  stay  ahead  of  the  fire.  Time  horizon  involves  the  team's 
awareness  of  two  time  elements:  upcoming  events  and  the  amount  of  time  it 
takes  to  accomplish  an  action. 

An  advanced  team: 

•  will  anticipate  upcoming  events  and  plan  activities  accordingly, 

•  has  knowledge  of  the  amount  of  time  it  takes  to  accomplish  an  action 
and  plans  and  assigns  tasks  accordingly,  and 

•  understands  lag  times  to  avoid  getting  behind  the  timeline. 

Time  horizon  is  different  from  time  management.  Time  horizon  deals  with 
reaction  time  while  time  management  deals  with  real-time  planning. 

Planning.  During  the  planning  stage,  the  team  will  identify  and  discuss  the 
amount  of  time  it  takes  to  complete  an  activity,  e.g.,  "remember,  it  takes  10 
minutes  to  move  X."  Making  sure  all  team  members  are  aware  of  and 
understand  these  time  issues  is  an  important  aspect  of  this  dimension. 

Execution.  During  execution,  this  is  evident  when  the  team  has  failed  to 
anticipate  and  work  with  the  lag  times  that  often  occur  during  real-world  task 
execution.  For  example,  a  team  may  start  doing  something  when  it  needs  to  be 
done  rather  than  taking  into  consideration  the  fact  that  it  will  take  10  minutes 
to  have  the  effect  they  want  right  now. 

Indicators  of  effective  performance. 

•  The  building  information  was  on  hand. 

•  Everyone  reacted  quickly. 

•  Information  about  the  layout  of  the  streets  was  shared  with  the  drivers. 

•  Members  of  the  team  anticipated  the  amount  of  time  it  takes  for 
equipment  to  arrive. 

•  The  BC  anticipated  the  amount  of  time  it  takes  for  equipment  to  arrive. 

The  value  of  this  dimension  to  team  performance.  The  commander  must  plan 
to  fight  the  fire  that  will  be  there  in  1 0  minutes  rather  than  fight  the  fire  he  sees 
when  just  arriving  on  the  scene.  Understanding  time  horizon  issues  will 
increase  the  likelihood  that  the  team  will  be  able  to  achieve  this. 
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CATEGORY:  TEAM  DECISION  MAKING 
Dimension:  Managing  uncertainty 

This  dimension  refers  to  the  team's  ability  to  recognize  and  communicate 
uncertainty  about  the  information  that  impacts  situation  assessment.  If  there  is 
uncertainty  about  the  nature  of  a  situation,  then  the  advanced  team  actively 
attempts  to  clarify  the  uncertainty,  develop  plans  that  are  flexible  enough  to 
allow  rapid  shifts  in  activity  as  the  situation  becomes  clearer,  or  hold  onto 
knowledge  of  the  uncertainty  until  it  can  be  resolved. 

Planning.  During  the  planning  stage  the  team  may  identify  what  they  know  and 
what  is  still  uncertain,  and  make  sure  that  appropriate  members  are  aware  of 
these  gaps  in  the  team's  knowledge  base.  Teams  may  also  assign  specific 
information-gathering  tasks  to  attempt  to  reduce  this  uncertainty.  During 
planning,  the  key  to  effective  performance  is  to  identify  the  gaps  in  their 
knowledge,  to  make  sure  appropriate  members  know  where  these  gaps  are,  and 
to  anticipate  how  they  might  gather  this  information  on  line. 

Execution.  During  execution,  look  for  the  team  to  occasionally  communicate 
not  only  what  they  see  or  know  but  also  what  they  don't  know.  It  should  also 
be  possible  to  see  team  members  remind  other  team  members  of  uncertainty 
when  appropriate. 

Indicators  of  effective  performance. 

•  The  BC  reviewed  the  Vital  Buildings  Information  sheet  as  he  was  going  to 
an  incident  in  a  building  that  he  is  unfamiliar  with. 

•  Information  was  shared  about  the  kind  of  buildings  that  surrounded  the 
address. 

•  Information  was  shared  about  what  they  didn't  know  about  the  building. 

•  Members  clarified  the  location  of  the  incident  when  they  were  unsure. 

•  Because  the  BC  was  not  up-to-date  on  latch-key  laws,  he  waited  for  a 
police  officer  to  arrive  before  he  broke  into  the  building  to  get  a  little  boy 
out.  Mieanwhile,  the  ladder  officer  noticed  fire  hazards  in  the  building, 
giving  them  reason  to  rescue  the  boy. 
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CATEGORY:  TEAM  SELF  MANAGEMENT 
Dimension:  Monitoring/adjusting 

This  dimension  refers  to  the  team's  ability  to  examine  itself  for  signs  of  both 
effective  and  ineffective  teamwork  behaviors  and  change  or  reward  behaviors 
accordingly.  The  purpose  of  monitoring  is  to  recognize  how  the  team  is 
operating  and  make  adjustments.  Recognizing  effective  behaviors  increases  the 
team's  ability  to  use  them  in  the  future.  Recognizing  ineffective  processes  and 
behaviors  and  changing  them  increases  the  team's  performance  on  line. 

Planning.  During  the  planning  stage,  observations  will  focus  on  how  the  team 
is  monitoring  and  adjusting  team  performance  within  the  context  of  the 
planning  task.  During  this  stage,  you  can  watch  the  team  to  see  if  it  is 
verifying  that  they  have  identified  and  communicated  their  resources,  roles  and 
functions,  goals,  and  uncertainties.  Also,  the  team  wants  to  monitor  members' 
engagement,  sharing  of  situation  assessment,  compensation.  When  the  team 
recognizes  inefficient  team  processes,  it  can  adjust  them  to  make  them  more 
efficient. 

Execution.  During  execution,  the  team  will  monitor  their  performance  within 
the  context  of  the  task  they  are  executing.  When  there  is  a  recognition  of 
inefficiently  performed  team  processes,  or  a  complete  lack  of  team  processes, 
the  team  attempts  to  reinstate  these  processes,  e.g.,  roles  &  functions,  goals, 
assessing  situation  assessment,  compensation,  etc. 

Indicators  of  effective  performance. 

•  When  the  BC  arrived  on  the  scene,  an  officer  told  him  what  they  had 
been  doing  and  what  they  intended  to  do.  The  BC  determined  if  they 
should  continue, 

•  The  BC  monitors  procedures  and  gives  direction  to  ensure  they  are  being 
followed.  (For  instance,  a  ladder  company  was  assisting  the  police  on  a 
search  for  guns  atop  roof.  In  an  assist  police  call,  the  ladder  company  is 
not  to  put  their  people  at  risk.  In  this  case,  the  BC  noticed  that  the 
firefighter  was  climbing  the  ladder  and  told  him  that  his  job  was  only  to 
usher  them,  not  to  do  their  job  for  them.) 


CATEGORY:  TEAM  SELF  MANAGEMENT 
Dimension:  Time  management 

This  dimension  refers  to  the  team's  ability  to  meet  goals  before  deadlines 
overtake  them.  Managing  time  includes  sequencing  sub-tasks  so  that  output 
from  one  task  connects  where  and  when  it  should  as  input  to  the  next  task  and 
allocating  the  right  amount  of  time  to  the  most  important  tasks. 

Planning.  During  the  planning  stage,  time  management  occurs  in  two  ways. 
First,  managing  the  time  of  the  planning  process  itself,  keeping  the  team  on 
schedule  so  it  does  not  run  out  of  time  to  accomplish  various  planning  tasks 
and  is  ready  on  time  for  the  execution  stage.  Second,  working  on  anticipating 
the  time  management  issues  for  the  execution  phase  such  as  attempting  to 
ascertain  that  they  are  being  realistic  about  what  they  need  to  accomplish  in 
the  time  available  during  execution. 

Execution.  Look  for  real-time  timekeeping.  On  schedule,  off  schedule,  been  at 
this  task  X  long,  etc. 

Indicators  of  effective  performance. 

•  BC  Aide  watched  the  clock  to  inform  the  BC  how  long  people  had  been 
looking  for  the  fire.  (For  instance,  if  firefighters  have  been  looking  for  the 
fire  for  1 5  minutes  and  no  one  has  found  anything,  but  the  smoke  is 
getting  thicker,  then  the  Aide  will  inform  the  BC  of  this.) 

•  The  BC  Aide  watched  the  clock  to  time  how  long  people  had  been  inside 
the  burning  building  and  reported  this  to  the  BC. 
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ASSESSMENT  FORM  FOR  OVERALL  TEAM  PERFORMANCE 


(The  team  consists  of  all  the  people  who  responded  to  this  incident.) 
Please  rate  the  overall  performance  of  this  team  during  this  incident  (circle  one). 

1  2  3  4  5 

Ineffective  Adequate  Highly  effective 


Please  circle  the  statement  (1,2,  or  3  below)  that  most  closely  describes  this 

team's  performance: 

1 .  This  team  could  perform  competently  during  even  the  most  difficult 
incidents. 

2.  This  team  needs  to  improve  some  of  its  teamwork  processes  in  order  to  deal 
with  more  difficult  incidents. 

3.  This  team  would  need  to  make  significant  changes  to  its  teamwork 
processes  in  order  to  deal  with  more  difficult  incidents. 


The  team  processes  I  would  like  to  see  this  team  improve  on  for  this  incident  are 
(please  list  below): 


What  interventions  would  you  recommend  to  improve  this  team's  performance 
(please  list  below)? 


ASSESSMENT  FORM  FOR  TEAM  MANAGEMENT 

Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 

MONITORING/AOJUSTING 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this  dimension. 
What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that  would 
indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 

1  2  3  4  5 

(Needs  a  lot  of  improvement)  (Adequate)  (Highly  effective) 

TIME  MANAGEMENT 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this  dimension. 
What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that  would 
indicate  '  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  RESOURCES 
Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 


MEMBER  RESOURCES 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


(Adequate) 


4  5 

(Highly  effective) 


LEADER  RESOURCES 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

» 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  RESOURCES 

Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 


NON-HUMAN  RESOURCES 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 

1  2  3  4  5 

(Needs  a  lot  of  improvement)  (Adequate)  (Highly  effective) 


BASIC  PROCEDURES 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  IDENTITY 

Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 


DEFINING  ROLES  AND  FUNCTIONS 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 

1  2  3  4  5 

(Needs  a  lot  of  improvement)  (Adequate)  (Highly  effective) 


ENGAGING 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

» 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  IDENTITY 

Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 

COMPENSATING 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 

Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  DECISION  MAKING 

Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 

ENVISIONING  GOALS 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 

1  2  3  4  5 

(Needs  a  lot  of  improvement)  (Adequate)  (Highly  effective) 

ASSESSING  THE  SITUATION 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

♦ 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this 
dimension.  What  were  they? 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that 
would  indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  DECISION  MAKING 
'  Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident: 

FOCUSING  ATTENTION  ON  RANGE  OF  FACTORS 

□  1.  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this  dimension. 
What  were  they?  _ _ 


□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that  would 
indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2  3  4  5 

(Needs  a  lot  of  improvement)  (Adequate)  (Highly  effective) 

FOCUSING  ATTENTION  ON  TIME  HORIZON 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this  dimension. 

What  were  they?  _ _ 

« 

□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 


□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that  would 
indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 


Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2  3  4  5 

(Needs  a  lot  of  improvement)  (Adequate)  (Highly  effective) 


ASSESSMENT  FORM  FOR  TEAM  DECISION  MAKING 
Please  check  each  dimension  as  present  or  not  present  for  this  particular  incident; 


MANAGING  UNCERTAINTY 

□  1 .  Present  (Check  one  of  the  following  two  categories): 

□  I  observed  behaviors  during  this  incident  that  would  indicate  performance  on  this  dimension. 
What  were  they? 

□  This  dimension  was  taken  into  account  prior  to  task  execution.  How? 

□  2.  Not  Present:  I  did  not  observe  behaviors  during  this  incident  or  any  other  time  that  would 

indicate  performance  on  this  dimension. 

How  was  this  dimension  important  to  your  team's  performance  for  this  incident? 

Rate  your  team's  performance  on  this  dimension  for  this  incident  (Circle  one): 


1  2 
(Needs  a  lot  of  improvement) 


3 

(Adequate) 


4 


5 

(Highly  effective) 


ASSESSMENT  FORM  FOR  OVERALL  TEAM  PERFORMANCE 


(The  team  consists  of  all  the  people  who  responded  to  this  incident.) 
Please  rate  the  overall  performance  of  this  team  during  this  incident  (circle  one). 

1  2  3  4  5 

Inoffectivo  Adequate  Highly  effective 


Please  circle  the  statement  (1,2,  or  3  below)  that  most  closely  describes  this 

team's  performance: 

1 .  This  team  could  perform  competently  during  even  the  most  difficult 
incidents. 

2.  This  team  needs  to  improve  some  of  its  teamwork  processes  in  order  to  deal 
with  more  difficult  incidents. 

3.  This  team  would  need  to  make  significant  changes  to  its  teamwork 
processes  in  order  to  deal  with  more  difficult  incidents. 


The  team  processes  I  would  like  to  see  this  team  improve  on  for  this  incident  are 
(please  list  below): 


What  interventions  would  you  recommend  to  improve  this  team's  performance 
(please  list  below)? 


Evaluation  Form 

Please  rate  the  usefulness  of  this  instructional  package 

We  are  interested  in  what  you  think  about  this  package.  The  instructional  package  and 
the  assessment  instrument  that  you  have  just  completed  are  still  in  draft  format.  We  are 
relying  on  your  feedback  to  enhance  the  package  to  meet  the  needs  of  firefighters.  As 
you  were  using  the  assessment  instrument  you  probably  had  some  reactions  to  its 
language  and  its  usefulness.  We  are  interested  in  what  those  reactions  were.  Please 
answer  the  questions  below,  being  as  explicit  as  possible. 

This  package  helped  me: 

(circle  one  for  each— use  the  back  if  you  need  more  room  to  explain): 

Understand  team 

performance.  Yes  No  Somewhat 

Please  explain. 


Improve  my  team's 

performance  on  line.  Yes  No  Somewhat 

Please  explain. 


Conduct  lessons-learned 

sjessions.  Yes  No  Somewhat 

Please  explain. 


Convey  team  performance 

concepts  to  firefighters.  Yes  No  Somewhat 

Please  explain. 


Please  rate  the  language  of  this  instructional  package  (circle  one) 

1 .  Appropriate  for  firefighters  2.  Not  appropriate  for  firefighters 


Please  explain. 


